powerbi - How to model queries in Power BI for daily append of new data instead of overwriting

Question

I'm trying to build a simple report in Power BI based upon data published on a website. Here is what I want to achieve

1. This website publishes data for COVID cases in the country.
2. The number are just the current numbers, without any time-series.
3. I want to fetch these numbers from this website daily and build a report on top of it (with time series kind of analysis).
4. So I fetch these numbers (Get Data > Web > URL) and get this into a query I then add a custom column with a timestmap (M's DateTime.LocalNow() function) and get this data with the required timestamp.
5. Now I want to refresh this query daily, so that I get daily results in this query. 6. As expected, PBI simply overwrites the existing rows with new data, with the latest timestamp (my custom column).

I tried few things like:-

1. Creating a new query and appending data to it, it doesn't seem to work, existing data gets over-written (maybe the way I have created the new query).
2. Explored incremental refresh functionality, it doesn't seem to fit my use case.
3. Tried looking at other similar posts, none seem to help me resolve this.

Questions:-

1. Is there a simple workaround to circumvent this (point#7) and have PBI append new data instead of overwriting existing data.
2. Am i correct on point#2 above (incremental refresh)?

Appreciate any pointers. Thanks in advance!

Answer 1

There is no simple workaround within Power BI.

Power BI is not designed to be used as a database where you store historical data. It's designed to connect to data and create reports from that, so you'll need to store the daily data somewhere external.

There are tons of ways to store the data. E.g., you could save them as CSVs in a folder that Power BI loads from or you could write them to a database table and connect to that.

---

Edit: That said, there is a non-simple workaround if this is something you really must do.

Though not recommended, you can use incremental refresh to trick Power BI into doing what you want.