So I'm going to start off with the aggregations feature. It's in public preview right now and it unlocks massive datasets like who here has seen the aggregations demo or some form of the aggregations demo.
You would have thought This is mutually exclusive, but you'd be surprised who here has not seen the aggregations demo.
There you go see it II always get surprised by this. The amount of the untapped market is astounding alright so I'm going to switch over to just give me one second here.
Alright so I'm here in power, BI desktop just another day at the office. Everything looks completely normal right. This data set is data for a crowd sourced Korea service where smart phone app emits. The drivers locations. And it generates a ton of data each row in this driver activity table.
Sense an individual location emitted by the app and this location. Count measure if I think zoom. It doesn't work on the virtual machine. But you can just about see up there. This is simply the count of the rows in the table. Nothing special at this point right so let's as I like to say casually.
First assembly rose we have in here right so at this point. I often ask the audience. How many rows. You think we have in it, like half a billion maybe a billion or 10 but let's push the boundaries. You know 10,000,000,000. You may have seen this demo already this is actually trillion if I go with this view.
You can see this the number of digits on this thing. This is a trillion roseanne. You just saw the counter. Osedax function right. This is a trillion rows of data. This is actually a quarter of a petabyte of data. I know y'all have petabytes all over the place. You have so many petabytes. You don't know what to do with them. I have so many petabytes now.
Sometimes I'm on my way into work and I see a petabyte here. A petabyte their snuff in this nothing petabytes are nothing to me seriously so you don't actually need a petabyte to leverage this feature. You know, some of you may just have half a billion rows a billion a few billion you may have a big Azure analysis, Sir.
That may be a model that compressed is at 100 gigabytes of memory that can be a challenge, not only is it expensive to pay for the hardware, but it's also a challenge to replicate all that data into memory so this feature helps significantly with those changes.
Is a quarter of a petabyte data? I'm just getting instant response times. I just got instant gratification and check out how interactive this is I'm going to break out distance traveled. I'm going to break it out by date. I'm going to make it a bar chart. This is so smooth. I'm going to break it out by the miles per job, I'm going.
Nice and big I'll go ahead and just filter it by the drivers who just left the company. I'm getting instant response times over a quarter of a petabyte of data just clicky clicky. Draggy droppy that are nice over massive datasets that you know it's absolutely seamless, absolutely beautiful experience. This may as well, just be a few.
Locally, in power, BI desktop right how on Earth is this physically possible if you know about power. BI and analysis services. We cash, the data into memory. We compress the data column oriented in memory database technology. We compress the data into memory, which is why we get this blazing fast.
But even without compression ratios quarter of a petabyte is quite a lot. You need a lot of memory right So what we're actually doing here is we're catching the data at the aggregated level only which unlocks these massive datasets in a way that was physically impossible before if you think about it. The 95% of BI queries that aggregated right, so you're going to.
Cache hit ratio right using a tiny fraction of the memory requirements right so I'm now going to filter on December 23rd and I'm going to break out the location count make it a grid break it out by the drivers.
Interactively, I record a petabyte of data created a list of drivers who works on December 23rd perform jobs of over 50 miles and subsequently left the company right up until now, I've been hitting the in memory cache. So I'm getting really fast performance and I'm now going to drill through to the detail records for one of these drivers.
To see the actual movements for Abigail Johnson on December 23rd so it's now going to plot a few 100 rose on the map from the trillion road table. So it's actually running a select query against the trillion wrote able to get this data because we're now at the leaf level, there is no aggregated cash.
Data is coming from databricks this runs on all of pretty much all of the direct query sources, including SQL data warehouse and so I'm going to come in to see this query that running right now and here we have Abigail Johnson and you can see this query is almost done already.
A selecting a few 100 rose from the 1,000,000,000,000 Rotable on Databricks. This takes you know 30 seconds or so on. SQL data warehouse. You'd be able to optimize the queries very 'cause. It would be much more targeted right. If you think about traditional direct query model that would generate a ton of queries on SQL DW not.
A lot more queries, but the nature of the queries would make it very difficult to optimize for ways. Here they're much more targeted so you can create indexes, etc to optimize for these queries much better, so I've now drove down and I've seen the Abigail Johnson Drive. All the way across Memphis just for Christmas. She got fed up, she left the company.
She started around company or something and she's doing very well, but point being we unlocked a quarter of a petabyte data in a way that was physically impossible before and this basically takes the cap off any scaleability limits that we had before, and it's way cheaper right because this model is.
How many gigabytes I think this is about compressed this is about 20 gigabytes of just the aggregated data and this is over a trillion trillion rises over quarter of a petabyte data right so I'm not going to go into how we set this up.
So any questions before II jump into this.
Feel free to chime in keep it interactive if you like.
Now, OK, I'm sure you come out yourself soon so up until July of last year in power, BI and an I services out services. Still, like this today. At the model level or data set in power. By 10 minutes. You have to pick pick whether it is.
Query or import right So what does that mean that means that direct query data set just acts as a semantic layer that receives the backs queries that are generated as users interact with visuals. It then translates those queries to the native SQL dialects of whatever the data sources and it.
Submits the query to the data source gets the data back and gives it back to power be. I write the performance of direct queries models models could be pretty slow but they had the advantage that you wouldn't use up a lot of memory and you wouldn't have to replicate the data into memory and you won't have to spend time optimizing whereas imports, which is the majority of the models tend to perform much faster.
You've got it, especially for these really large datasets. You've got the challenge of fitting all the data into memory. So then we introduced this feature called composite models, which allows you to not have multiple direct resources, but allows you to combine important direct query in the same data set so that you get better bang for your Buck or?
I can't think of the English translation right now, I've been living in America 12 years. So you know the kick in for pound or something. I don't know the bank for your Buck for memory consumption right so that you can pick which tables to cash into memory where it counts so this was a useful.
We introduce the aggregations feature on top of it, so if we pretend that sells has 10,000,000,000 rows and we create an aggregated table called cells AG right? Which in this simple case is just the nonempty cross join of date. ID angiography ID, which is going to be a few thousand rows or maybe a million or 2 rows.
Much data at all this, a big super easy to fit into memory right and it's just a sum group by cells of those those keys and then when queries get submitted from power be. I so here's. The thing that sells AG table is completely hidden in an upcoming release it will have to be hidden.
Add table is completely completely hidden and the queries. The users know nothing about it. So the query. Just ask for some of sales and it's asking to aggregate by year and city which happens to be attributes of the 2 dimensions that the cells AG table relates to.
And therefore these can hit the aggregation and and the base of the metadata that you set up which I will show you in a minute. The engine internally re directs that query to get the data from the in memory cells. AG table and the user doesn't know anything about it. It's completely abstracted from the user. The only impacts on the user is that the query.
It turns really fast right then the user drops in instead of the an attribute from geography bee user says well. I want to aggregate by customer name. Instead, right at this point, it cannot hit the cash. It can't hit the aggregation be'cause its grew.
Customer name and for a particular Roman sales ad. They'll be one row in geography. But there could be multiple customers. It would know which customer to aggregate to it, so sorry. I can't hit the aggregation and then it reverts a direct query. Just on the fly. Just like that right there's no separate measure. There's nothing. It's completely abstracted from these it just on the fly reverse.
I haven't got a cash. I'm not going to get the data straight from the source 'cause. I'm now that detailed level and you'll notice that the dates table. If I go back a slide when I'm hitting the in memory cache that date table is dark blue to represent that it's in memory right whereas when I hit.
Don't hit the cash. It then turns white to represent that it's in direct query becaus. It has this ability to flip back and forth between important directory has this jewel function. It uses the new jewel storage mode that you set at the table and this is important that these tables.
Back and forth between important direct query because if you think about it if they had to stay its import and it couldn't switch the direct query on the fly and in this query would conceptually have to basically get all 10,000,000,000 roads of sales and then do a join on the power by side to the dates data that is in memory.
It's much more efficient to just push that that join operation that some group by to the source right way way way more efficient. So, therefore, these tables can dynamically flip back and forth between important direct query on the fly depending on the granularity of the query. Does that make sense that's the conceptual explanation of it and I'll now.
Any questions about this yes go ahead.
Yes, thank you. Dan you the the question was can you have multiple aggregation tables for the same detailed table? Yes, you can so that brings up a point about like? What is the best practice? Should I have one big wide table that covers lots of my dimensions?
So I have lots of little ones. I think generally you should start off with one big one and potentially you could break it out. There's another good use case for that which is the aggregation table doesn't actually have to be stored in memory. It can be direct query as well, and you can optimize it using column store at the source etc. So you could have.
Smaller aggregation table that's in memory. That's for your executive dashboard or something and then you've got a big wide. Aggregation table that isn't direct query that might be optimized at the source, maybe using column store or something that is lower granularity and I will show you how to set up multiple aggregation tables. If I remember basically there's a press.
Property in the aggregations dialogue that let's You set that up and then if I had multiple aggregation tables points at the same fact table. The one with the higher precedence would be considered first my sense, so yeah, good question any other questions.
Don't be shy. I want another question Alright OK. Let's move on. So I'm going to show you how to set this up now.
And I believe this is the one.
So this is just a simple little model right, I've got myself table, which let's say has 10,000,000,000 rows. This is very close to the conceptual model. We were looking at in the paradise in the PowerPoint slides and this is my aggregation table right and.
II set up this model just like there's nothing special about this table right now, it's hidden. But it's just another table right. I created this table just using the standard get data experience like I did. The other tables. So why is it just another table if you know aggregations in multi dimensional they're kind of baked into the mall?
Where is this is just another table because it gives you a lot more architectural options this way right you could as I mentioned this could you could just leave it as a direct query table you could optimize it using column store at the source it could be just a massive aggregation table that you can't fit into memory you can use incremental refresh.
Mental data load process is in your data warehouse or you can set it to import and then if you set it to import you could you could even perform the aggregation on the fly in the query behind the table or they probably wouldn't buy that's an option and then once its import you'd be able to.
Like incremental refresh on it, it's in direct query mode at the moment, so it's disabled but you know it. Just gives you a lot more options, especially for working with externally stored and externally defined aggregations right? That's why it's just another table alright so this table.
Prepared in my data warehouse and it's the sum of cells grouped by customer ID sub category ID and product category. ID customer ID date ID and Sub Category ID. It gets to product category automatically so it works with like the Snowflake.
Models right so it's going to have a lot less than 10,000,000,000 rows if you think about the nonempty cross join across those 3. I DS is not going to be that much data and it would be pretty easy to fit that into memory so first thing I want to do is set this to import.
And it pops up, with this message here, which is saying if you're going to set this table to import. It is strongly recommended that you set these other 5 tables to be jewel right so if you remember that thing where the date table was flipping back and forth between important direct query right so it's saying, You're going to get.
More optimum model if you set these to jewel you know it is going to have to catch the data for these dimension tables, but generally the amount of data in the dimension tables is a lot smaller and that means that whenever you do an aggregation of a metric in the aggregation table by any of the attributes in these related dimension tables it will all.
Main memory right so it's recommended that you leave this checked and I'll click OK and it will now. Refresh those tables. So if you have a big model, maybe with a 50 or 100 tables and you bring them all in from your data warehouse and it's all direct query. Initially and there's just a couple of tables that you want to cash into memory you.
Figure out what are the minimum dimension tables that it needs to set to jewel rather than you doing itself, so you'll notice also that I'm using the new modeling view. So if I wanted to set the properties. Myself I can multiselect. These tables and I can set the storage mode. These are the 3 storage modes at the table leg.
Start Korean jewel right and I can multiselect and set them multiple tables at the same time with this new modeling view that we launched recently right which is specially for these large models right so everything is set up now except the actual aggregations themselves.
So I'm going to come into the manage aggregations dialogue and so here I've just got all of the columns here in the aggregation table right so I've got the aggregation table. Selected and I've got all the columns here and I'm going to leave the foreign key ones alone for a minute.
Set up the actual aggregation one so this is you know, I would have prepared this ahead of time in my data warehouse brought it in as a regular table and then now. I'm going to say OK. This is the account of all of the rows in the table, so there's a special one in here that is count table rows and I say this is the count of the Rose in the sales table and then I don't get to.
Spell column because it doesn't, it's just account of the Rose. Then the next one. I'm going to do is sales amount. Some so this is a sum so these are these are the aggregation functions. Some Min Max Count of table rows and count and group by but you'll see that you get a lot more coverage in terms of the dax functions, which I'm going to show you in just a second.
Inside this is cells amount some from the cells table and.
Define sells amount there, it is, and then here I've got 2 aggregations on the same detail column on the unit price column. But once accounts and the other ones are some and you'll see why. I'm setting this up in just a second so this one is the count of sales and unit price.
Find it there, it is.
And lastly we have some of sales and.
Unit price right so I've just set up those aggregation mappings now set up all the metadata. I've set everything up right. This is much easier to understand than aggregations in multi dimensional just a fact that you've designed this data model means you got a pretty good idea of what queries.
Going to hit your in memory cache right. I'm now going to switch over to the great backs studio and I've this is this query here.
Is running a sum of cells by calendar year? I run it beforehand before I set up the aggregations and you can see that it ran a direct query for this and it hit the detailed table fact Internet cells and I'm going to run it again. Now that I've set it up and I would expect this to hit the cache becaus.
Calendar year column is in the data table, which is one of those related dimension tables that we set to jewel and if we see here. We gotta scan and you don't need to worry about the XM SQL Syntax. But this is the storage engine query and you can see that it did actually hit the Aggregation.
Then here we got this nice aggregate table rewrite events that says, we got we got an aggregation hit right so you can use this to check whether your queries are hitting the aggregation or not then.
This query we would expect to not hit the cash. In fact, maybe I could put this to the side and we can see the data model in the background.
So this query is by doing a group buy English product name and that's in this table and this is not at the granularity that can hit the cash so therefore if I run this.
We get a sequel event. Here's the direct query. If I went there. We'd see that. That's fact in itself, so it didn't get it from the AG table and the rewrite event says sorry you didn't get a cash hit so this is just working as designed and then now I'm going to run this one this is account.
Are we set up the count of table rows again? We get a scan and a cache hit so this? Is working as well. And then here. I'm going to run an average now you thought I wouldn't be able to get an aggregation hit for average on the face of it. But if you think about it. This is actually doing an average of unit price, which was that.
Both some and account for right so actually I should get a nag hit.
And I do and the reason is that the engine kind of internally folds an average into some divided by account setup since I set up aggregations for both the sum and the count on that unit price column. I got an aggregation here, even for an average right so this is trying to send to show you that we get.
Backs aggregation function coverage than just the sum Min Max Count and count table rows and then the next one. I'm going to do is distinct count and so you may think well that should be impossible well. Currently, it is 'cause. I haven't done anything. So we're not going to nag hit so if we look here. This is the.
Query right so you can see it did the distinct count there. Now we can actually in this particular case. We can actually get it to hit the cash So what I'm going to do. I'm going to come back here. I'm going to come back to manage aggregations and that was the distinct count of the customer key now customer key is.
The foreign key columns that we've got in the table, so we can actually set this up as a group by you know the group buys for these kind of dimensional models where you rely on the relationships 'cause. It's relying on the relationships to get the aggregation hits the group buys are kind of optional.
Here for a distinct count. I do need to set this up so I'm going to say this is a group by of sales and customer key.
And if I save that and then back over here.
So as you can see that's a distinct count customer key.
And I now actually get a nag him even for a distinct count, so the reason this works is that I'm telling the engine. The you know, I'm flag in the customer key attribute as a group by column, which means that if I can group by its that means I must have all of the distinct values in it. I'm maintaining the distinctness or the uniqueness of the customer key in the.
So I don't know 5,000,000 and we're looking at other ways of addressing that. But in this particular case. I could I can avoid a direct query. Distinct count and I can get the distinct count from the aggregation table right and down here. This query is just the same.
By the count of the roseanne I get a nag him. We would expect it because we've already seen someone count rose work. But the reason I'm showing this is that it works for it's not just for simple some measures right. It works for complex calculations and complex measures that will take the sum divided by the count you know multiplied by the percent of the para.
And all of these complex queries. They ultimately get kind of folded down into a logical query plan and they'll be a bunch of these sub queries that that get submitted to the storage engine and for each of those subqueries. It will say. Can I hit the in memory cache for each of those subqueries it will say is it at the granularity that can?
And if it does if they are then then it will so this is in this case. I've got 2 subqueries right, but in the average example. There was also 2 subqueries even though there was only one explicit. Dax aggregation function right any questions about this is going to be some questions yes.
The question was is this exactly the same in analysis services no so you know.
This is leading to what I'm going to talk about coming up in this session that we have these 2 products power BI and we have 33 product 6 overnight services Azure. In those services, we have focusing at the moment on bring in the analysis. Services functionality into power BI like we can't just light up every feet.
Products cause all of the tooling is different there's a ton of extra development work, etc. The whole part 2 of this session is all about the new features come into analysis services so there are a lot of good new features coming to an aid services. This isn't one of them at this time.
Focusing on bringing the analysis services functionality to power BI at this time and I'm going to talk about that next yes go ahead.
Yeah.
OK so the question was can you create separate measures on the AG table and the detailed table? Yes, you can and, technically, you could do that today right the only thing that you may be missing is the composite models thing. But the difference is that that that means your users need to know which measure to select.
Sexualizing, you know this is my measure that I use at this granularity. This is the measure that I use the detailed granularity and the user experience is not a seamless, but technically you can already create separate aggregation tables with separate measures today any other questions yes.
Excuse me it count rose count rose.
How is count rose hitting the cash right becaus? When I when I create this aggregation table if I zoom in here so I create this aggregation table. Let's say I create it in my data warehouse and I say I want the sum of sales and the count of unit price and blah blah blah and there's another column that.
Let's have populate it using a SQL query and there's another column, which says just give me the count of the table that I'm aggregating is the sales table just give me the count of the rows in there and I store that output in this column right here in this column right so this column has the count of the.
And the the cells table and then when I set up the aggregation mappings.
I said here for.
This guy I said, this column fact Internet cells underscore count holds the count of the Rose in the sales table, so set up account. Tableros aggregation mapping for that right. I don't specify a detailed column. I just set up like this and now the count rose function, referring to the detailed table.
Sales table is going to internally the engine will re direct it to the aggregation table.
And other questions.
I'll take one more yeah.
Technically, you can, but you would get a lot of re calcs on a big table. Quite frequently so the question was is there a reason. You can't create the aggregation table using dexas a calculated table and in the early versions. You weren't able to now you can, but for some cases that might be useful, but you would get you.
You would every time it recounts that which would be quite often. It would take a long time. 'cause it have to query. The big detail time I'm going to take one more question yes.
Let's see so we got the count table roseanne count.
Yes, thank you so the question was what's the difference between count table rows in count so Cal table rows. I do not get to specify a detailed column and it's a pre aggregated precomputed count of the Rose for count table roads, whereas the count one is the count of.
Column right so if I do the dax count function on the unit price column on the sales table. Then then that would work so that would otherwise be 2 different numbers right. I'm going to move on. I've got a lot of content to show you.
Alright now I'm just going to highlight briefly that.
When you set up the relationships between these tables just let this come up so technically you can create a relationship between an import and a direct query table. But in most cases, it's probably not recommended for especially for like enterprise models so I would call this.
Relationship why is it week so it will not get aggregation aggregations will not observe those relationships for AG hits and it won't. If you think about it again. It would conceptually have to get all of the data from the 10,000,000,000 rows in cells and do it join on the power by side to get to the data table.
Practice it, it runs in a query like based on the privacy levels in power behind it runs and includes web filter. It's not quite as bad as that but it's still nowhere as efficient as if it can just switch that date table to direct query on the fly and push that some group by operation to the source right? That's way more if.
Plus it will be observed for aggregation hits right so I just wanted to mention that and here are the rules basically anything on the one side of one to many relationship if it's jewel it. Will it will be a strong relationship assuming it's a single source right now, then I'm going to briefly show?
If I'm just going to reopen that VM that just got killed.
That.
You know the the.
The model we were looking at was kind of a dimensional model right. We had it was kind of a snowflake and we had relationships between the tables and it depended on the relationships to get the aggregation hits now actually. I don't actually need that VM. I don't think if I've got yeah, I can just show you.
This is what the The Model looks like that was getting data from databricks right, so from these spark sources.
Thank you. Thank you thank you. Thank you. Thank you for help. Alright so these spark sources like these big data sources. Those models tend to not they tend to avoid joins right because they don't deal with joins as well as like SQL. DW would for example, because ultimately these are stored in a file system. These are base.
So you you tend to get these highly denormalized models where all of the dimension attributes are effectively deemed normalized into the fact table and so here I've got this is actually the table from the spark source with a trillion rose.
Aggregation and this is prepared in spot. This has about 1.68 billion rows and so we got we got rid of the Lanja Tude and latitude, which reduce the those are the high cardinality columns and we have reduced the rowcount significantly so this is the aggregation table, but how do we set this up because there's no relationships right?
Little bit different, but we did it in a way that will work for both of these models so if I come in to manage aggregations here where before the group buys on the foreign keys were optional except for the distinct count now. I need to set up a group buy for every single one of the attributes that I need that I want to aggregate by.
Freight to to know that it's going to get another hit, so down here. We've got a couple of the real aggregations right. These 2 and everything else is just an attribute with a group by and all of the dimension attributes are covered. It set lanja tude and attitudes. Are any query. That doesn't include Lanja Tude and latitude should hit the cash as soon as I drop in Lanja Tude in LA.
Map then it suddenly reverts a direct query right and you can see here we got the precedence as well so back to the question earlier that Presidents Field is what we would use if we wanted to set up multiple aggregation tables on top of the same detailed table and other questions.
Now, OK, I'm going to move on now, so that's aggregations that you know really will take the limits of Scaleability. It gives you a much more balanced architecture. You know you might have all of the data prepared there in your?
And then you can protect the concurrency limits in your data warehouse. You can catch 95% of the queries. The queries that do get through to the data warehouse can be optimized much more easily. 'cause they're much more targeted Alright. So it gives you a much more balanced architecture and is way more powerful in terms of unlocking these massive datasets alright so.
Back over here.
I got 10 minutes left.
So.
Traditionally, we and our competitors and reset agencies and the industry would kind of separate enterprise in self service, BI Rite. We've got rich heritage in enterprise by another services is 20 years old, where the only vendor to be in lead.
Turn in Gardners magic quadrant for 12 years now. You may have seen the latest one that's before self service by even existed as a discipline. It shows the heritage that we have and where the only one of the traditional by vendors that embraced self service by early but what we end the industry is starting to reel.
You know the life cycle of a by artifact isn't as clean cut as just enterprise or self-service bite. Sometimes you know it's somewhere in between or you may have an analyst discover something but then once they get promoted to enterprise BIT wants to take ownership of it. They don't have to reinvent the world. They don't want to lose the essence of it. Obviously, there's going to be some work.
Make sure that the naming conventions aligned with corporate standards, etc. But it's better. It just it just makes the whole collaboration between the business and it it that much more seamless facilitates that and it and it just helps if you colocate all of your by artifacts in a single location.
Sumption standpoint, you know everything you don't need to go to one interface here and one interface there. Everything is just Co located and it just gives you that unified architecture, So what we're doing is we're bringing the analysis. Services functionality to power by now. The reality is that it's already there right power. BI datasets run on an honest.
In the back end, So what we're doing is exposing the analysis services engine in power, BI and we're doing this by opening up the XML a endpoint so the XML endpoint was first launched in a nice services 2005. It was defined of a few years before that, it's.
Pretty standard there's multiple vendors that use it basically every other Tom Dick and Harry or every other major by vendor on the planet every other major vendor in Gartner's magic quadrant supports connectivity to analysis services. Becaus and as services uses this XML a protocol right so you have all of these client tools.
It says to the United States is server name right so that that is includes. Microsoft tools as well as then management tools, debugging tools modeling tools. Third party tools. Tabulated heard back studio. Bisson normalized there. There's a ton. Everything is based off this XML endpoint right. It's how you how you get access to.
Serves his engine now since power BI runs on and our services. We it's not a huge stretch for us to just open up the XML endpoint in power be. I which means that you get all of that all of those those tools. In all of that functionality just works with power BI as well as the pro.
So a lot of the tools that I mentioned that they will use the client libraries that we have for analysis. Services, which gives you the programmability. The tablet object model. You can traditionally you would use it quite often to create and manage partitions. But a lot of the third party tools use this API very rich API that Apple object model you got scripting.
Is tablet mode of scripting language that you can run from 6 server management studio you got some powershell commandlets that Azure analysis. Services inherited from SQL server analysis services so all of this is based off this XML endpoint right so if we open that up, then there's a whole host of functionality that you basically.
It takes a big step towards making power by a super set of an hour service there still going to be a few things on fringe cases on the side like backup restore and scale out, etc that we are actively working on separately. But the bulk of the functionality. That's missing from power be. I that's in analysis services will.
And the other one is big models right agonized services. You can have a server with 400 gigabytes of memory. There's a cap of 10 gigabytes in power, BI premium. We're working on lifting that so overtime in the not too distant future. This will get read read only access to the XML endpoint will get announce pretty soon.
And then the demo that I'm going to do, which is probably going to be after the break at this point is going to use incremental. Refresh and XML combined in the same demo so I just thought I'd tell you about incremental. Refresh so in fact, the Friday.
Are there any questions about the XML endpoint?
Feel free temperature, yes go ahead.
Yes, so thank you that the question was is XML endpoint on premium only so the answer is at least initially. Yes, for read write operations. It always will be for technical reasons. We do want to try and have a symmetric at least modeling surface area between premium and pro struck shared.
Aggregations in composite models working both incremental refresh. Currently, only works in premium, but XML point will be launched as premium. Initially, one day we'd like to open up read only access in shared or pro. But it's just going to take time. And as I say there's a cost to doing all these things that we just need to.
OK, yes go ahead.
Sorry was the question is incremental refreshing to show up in pro.
Yeah, we've kind of publicly stated that it will sharpen pro. The reason the incremental. Refresh is still in public preview is that there's a few things we'd like to do, we'd like to get it in pro? We want to do the metadata only publish right and a few other things, so yes.
Get it in pro again. It's fits into this category of having a symmetric modeling surface between the 2 technically there's no reason why we can't. It's just extra work and we just haven't got to it yet, all right in other questions yes.
OK, the question was X-Men point? What's the supported authentication method, so it depends if you access. A local instance of an eye services running on your laptop. It's using XML endpoint. If you access Azure. No services it's using XML endpoint or power, BI premium now it.
So the authentication method is going to depend on what product you're connecting to right so if it's on Prem. It will be Windows Active Directory, where whereas if it's Azure. Analysis services or power. BI it will be Azure Active Directory, an oauth and uh.
Pretty much done here alright so I've got 3 more minutes. I'm just going to conceptually explain incremental. Refresh so they deal with incremental refresh. I mean, we've had incremental. Refresh in analysis services for over 20 years. The first version of another services, had partitioning and so this is what?
In an analysis services for is that you know if you have a large model. You don't want to have to reload. All of the data every time you do a refresh. Right or in a nice service terminology every time you process. The model so if you are going to do a refresh. Every night and you have 10,000,000,000 rows in 10 years of data, it's much more efficient if the data the only data.
The last few days you should just refresh. The last few days. It shouldn't need to refresh. All 10 years of data right. So so with analysis services. You have to write a ton of code to manage all of this yourself using tablet object model or other APIs for other versions of analysis.
Whereas with incremental refreshing power BI. It's just a dialogue. You fill in this policy and then the engine generates all the partitions for you right after the break. I'll show you the will go use the XML endpoint will connect to power. BI premium will look at all of the partitions that are generated through SSMS all of the partitions that are generated.
Going to refresh right and then we'll get onto the big announcements that are going to ship that are shipping today. In 6 overnight steps is 2019 and will also ship in Azure Analysis Services and power BI OK. So we'll catch up after the break. Thank you very much.