OK, everybody, so I think we'll make a start.
As a slide mounted Skaven. I'm going to be talking about disposable development environments with Azure Devops.
It's been a long day, and this is the last thing before the party at 10 o'clock conference and I know it's hard at the end of especially if you're spending your money. All the bosses money or the boss is sitting next to you here right and you feel like you have to sit and absorb all this stuff like you know 8 hours of watching sessions later can be quite.
I can be quite draining actually just sitting in the audience for that length of time, so I do appreciate all of you.
Hanging in there.
Well.
Article 3 different things. I've done quite a bit of database. SQL E&BIE development type stuff in the past that application development.
These days, I mostly work on.
Getting a software that other people have shipped into a place where you know users can use it and pay for things and so all this stuff about delivery pipelines and automated testing and infrastructure as code, which is really what we're going to be talking about today so.
Actually Genesis of this talk is.
Something from a couple of years ago and.
Why are we talking about this at secrets now? What's funny actually when I was writing this slide so we're going to look at the program for SQL bits. In 2016 and see what they were talking about then and there was all this stuff about big data and that was really the thing and we had big data. We have data factory had data Lake and data warehouse and it was all about big data but I think they have.
Stations of all these things but I think that one was really saying at the time.
Is it actually no customers were using that stuff in production right now? Fast forward like 2 or 3 years?
There are customers trying to do like real work with data Lake and data factory and all this stuff and there coming into the kind of challenges that we say in like all kinds of software like? How do we get our work that we've done in a place where it's delivering value so that's really why this is kind of relevant now not only about data actually.
These contact makes about you know disposable infrastructure. These are 53 kinds of software development anything where you've got like a pass service. Even though our service. How do you manage that development thing of working together in an effective way?
So data factory.
I.
Already be expert in data factory kind of everything I know about it is on this slide and.
I think when it was initially introduced it's kind of being marketed certainly to some audiences being like.
Well, this is like SIS for the snapshot generation. That's kind of turned out to be a mischaracterization ready right because this tool is not really in this size space tours in the space of these workflow orchestration tools wish there was some, like you know enterprise, he ones like you know.
Autosys or control M was it called and those are like scheduling tools and then there was some more data specific ones, which are mostly open source like ask around or like Luigi or like the other one air flow and this is the space that data factories really units, tying together. All these different things that you might have in your enterprise these data sources and.
Processing resources and orchestrating them to ship data and insights to the end users in a reliable way to do this is a couple of things that go into a data factory. They just want to touch on this is going to come back again right so.
Adult factor is really just like a container of stuff, maybe like a security boundary of resource boundary and it contains these things called pipelines and a pipeline is what stitches together, the activities that we do is part of ELA dataflow.
And apart fun contains these things called activities so in activity could be copy. Some data from one place to another or you know, do some processing to enrich the data that we've copied.
You know things like that, and these activities have a couple of dependencies right, they have to have some way to run so you know to run a SQL server store procedure need to SQL server, and for instance, to run a hive query. You need to have a Hadoop cluster.
And.
Datasets so In addition to having this compute resource where we actually do the thing the activity.
A lot of the data that we deal with in these scenarios is what we refer to as unstructured data right. But we know that unstructured data is not really unstructured. It's kind of unstructured when it's at rest and these data set is how we define the structure that were either reading from or writing 2 overtime of doing the activity.
The data set is kind of tightly coupled to the data that you've got stored and to the activity that's kind of a link.
So I have a little example project. This is the kind of thing you want to do with the data factory so Orwell label on the left. We got a blob store. We're going to copy a file into a data. Lake store right so this is going to be far I'm going to do some processing with.
Data Lake Analytics. As the use equal thing and then insert the data into a SQL data warehouse.
Already going to go into the specifics about any of this stuff works, except to say that these are facing some things we can do in the data factory is kind of orchestrates all ties it together.
So when I first started researching this online. It's not right. So the way we do data factory development is through the portal.
And the fourth is coming out this gateway into another dimension isn't it like this. In this case. The cloud so? How do we go about developing for ADF in the cloud?
Well, I gotta animation here, so we can drag and drop dragon, dropping is always good. We can drag and drop a task onto the canvas there and then we got a bunch of these properties dialog. So we can fill things in right. So we're going to set up here wherever data is coming from.
All this connection stuff was already set up.
And I'm going to give a path to where are far is right so this is the source data there going to be popping?
And then we can get a preview of our data set here.
And here's the preview right This is from my.
Stealth startup which is like over, but for kabobs right right.
So here's our data. They're going to come here and we can imagine working away dragging and dropping all morning in this portal.
And now it's time to go for lunch, possibly even Frick about there's an app for that, said, what we want to do before we go for lunch.
We want to say is what we've been doing.
And once conspicuous by its absence from.
This editor here is a save button.
There's no way I can click to say right save what I've been doing.
Let's publish so publishes right take what I've got here in the designer and put that in a data factory. But we want to be ready to do that. Yet we're working? How are we going to save this data so we can come back to it later or our design so we can come back to it now there isn't a safe but there's an export.
It's here exporters arm template going back to arm templates in some detail right. But for now, and on template is.
A file that represents you know the whole design of our data factory and we can export it to a file on our local hard drive through this interface here.
And then we can import it later.
And there's a very large number of parameters that we can supply. This is a really good example. If configuration over convention anything that could possibly be parameterized at deploy time is in fact, parameterized by this tool so not only can we specify in our import files or output files. We can say what as your tenant were using what account we're using?
And all these things will be passed in as parameters.
So they are servicing too late. Actually, too big in this presentation. But on stackoverflow this week. There was someone asking a question about this and eat exported a template that had 293 parameters here in the dialogue and then couldn't import it because there's a limit. I want to parameters you can import when you report it so he?
Use this tool to create a non round trip abal data factory.
I've not answer the stack overflow question yet unknown as anyone else, so you can jump in anytime weather.
And also yeah companies from his red red was that 'cause that secret, so these contain things like you know database connection strings or?
Azure storage connection strings can be sensitive to, if the authentication is being done through the token in the connection string and.
Obviously, we don't want our developers to have access to real potential for production right and actually original custom was doing. This sort of solution to this, which is like OK, so we're going to export our.
Play factory design, some files with a tool like this and then we're going to another team that has access to the production credentials. So that's going to importing through the portal and type in that creates greater and guess? What my name is the other team was.
Obviously it was a devops tamer right labor were going to dragging and dropping in the typing in so anyway. So we're going very well and we've been developing our packages in the factories in the portal and exporting importing them.
What happens when we want to grow?
What happens when we need to scale out the number of data factory developers we've got emetine?
I'm going to get to that robust about 10 slides away.
Sorry.
Image we've got 3 developers work I'm assuming factory and you know, one of them adds a couple of tasks to their canvas and then this one here comes along had some different tasks to the canvas and maybe the floor is that the developer. One doesn't know about this so they carry on, adding stuff to the canvas. We got a spark thing a service.
So what's going to happen when developer one deploys what they've got the data factory. Obviously is going to wipe out developing number twos changes.
So what's missing from how these people are working well. We're not doing that, not taking each. Others changes 7 are integrating with the other developers and how do we do integration? When we're writing software? We do it with source control. That's funny. I think the old days right. This wasn't such a thing because.
You know, these belief systems like is in particular, you know they say the analysis services type things they were done by like one person sitting in the corner and no one really knew what that person did and I can see some people here used to be that person and maybe some people you know, some application developers. They might like a website and then they'll be some like SQL people do.
Mysterious character who was like in charge, Cube, but it didn't really matter how he was working.
No one else really involves going on there. So yeah, that person didn't really need to have source control really right because they were never integrating anyone else is work.
I but now that's what kind of change right now, these days. Everyone does everything we all have to understand how to work together in a kind of good way.
The source control is really the thing that allows us to scale what we're doing beyond develop one developer.
Everyone is cool with get right.
Now.
Missing her so people think that gift is this horrendous Lee complicated thing for actually.
Get is addressing a pretty complicated problem space.
But what we know, people smarter than me. You've done in the past right here right so get is solving this huge horrible problem space that we can simplify that problem space by adding these workflows and we can work in some like let's say predefined ways. And they can be get flow. There can be github floor. You get love flow. All of those things right what all those.
Or about is let's take this really complicated thing and make it a bit simpler. So we got some rules to follow and accordingly reduce the amount of the surface of git that people need to understand our only work.
So this is kind of a simple workflow and so the green blobs across the top that's the Master Branch and we got some commits to each of those is the hash and then 965, there we say writing to a new feature and we create a feature branch and then you know developer writes code and doesn't commits and then there's a pull request and the problem.
Outside the gate process here, which it is right, we say to someone else between leader peer review or whatever. This code here. I'll go in the Blue Branch. I want to merge into the Green Branch, which is master so we can ship it and then when that's done.
We update the Master Branch at 956, there with everything that was already in the Master Branch and everything was in our feature branch in blue and we can get rid of our blue branch now data factory has get integrated.
So right here from within the data factory why we can do this setup code repository and the next slide is gotten awards on right. This is from the Microsoft documentation about continuous integration delivery with data factory why I'm going to cut out some bits are.
So talk one.
Set up my development data factory great without your repost so without box.
In which over developers can offer data factory resource is like pipelines and datasets right right. All that changes. And they can debug debug funnier so debug doesn't deploy but it uses the real inputs and Outputs, a real files so has the real dependencies, but not to build data factory.
After it's all working we can create a poor question knowledge. It then we publish it to the development data factory.
Uh.
When the team was to promote you can export like we saw in very import with the parameters right.
So.
Is there anything wrong here?
What's it like developing in a shared development sandbox, we used to corda?
So it must be some people here who.
I've done a lot of SQL development on Prem before or BI development on Prem right? How much fun is it when you share a development database would like every other developer team will work places like that right? Yeah.
If you think about everyone application developer everything to C Sharp program where Java program. Yeah, this is like saying Oh, yeah, so I'm going to do my work here and I bought my code. But last time something from the other side of the office is going to like remote into my machine and monkey Patch deals that I'm debugging. It was crazy. But for some reason for a database development. That kind of thing would be debugging something.
The state of the thing we're trying to make work is changing behind us.
Right.
I'm really sorry that that whole thing right. This shared development environment. This is a terrible idea on. But in order to ease the journey to the crowd to the cloud. We're going to take that terrible idea from on Prem and lift and shift it to data factory.
Someone is this really like in practice, so another little video. So we're going to get something out of Design Services AU SQL task. So we're going to run by you, SQL script.
I cannot write a script is stored on this is where we're going to run it in our data Lake Analytics Service.
The script is stored in external file I'm going to give up after this file. That's also in blob storage and I think we're going to preview with the file in a second.
So this script is not doing anything, particularly exciting. This is like standard ETL stuff. I think we're reformatting a date and then reformatting the chili and garlic source into a junk dimension.
Is not about ready to create a new branch? We can uh get our changes merge to master but we can't create a new branch until we save their changes.
So we collect safe.
Was it save my changes?
That's right side my changes to the master branch right right and it saved them with this great commit message here that tells me exactly what I was doing and why right so that would be really useful like you know for the next guy that comes along to figure out why the data factory works like this and of course, it wasn't a master branch right. We didn't have to create approval. Crystal we could just committed to master.
Now, many systems allow us to protect the Master Branch. I guess you must be doing this at work right. We say from Master Branch? Is yeah, everything has to be through PR with some reviewers or everything has to have a test build so we're going to protect the Master Branch way to stop those pushes director master.
So now, what happens is going to make a change to the data factory.
I'm going to try and create a new branch to commit it.
We can't create a new branch because we have to save first let me collect safe.
We got exception, so really this developers, losing their work right. I can't connect it to the feature branch. They didn't create the feature before I started and I can't commit to master so by simplifying how it works. We've made it easier rather than harder to like lose what you were doing.
This is what the Reaper looks like behind the scenes right so these are all the files that you I created the bases save the state of our data factory as it wasn't designer here. So we can recreate it. Yeah wanted to publish something else happens right. This other branch appears in our report this Blanche here.
So of taking what was in the master branch to fly into a database and extracting it again.
Last ones were all but this is really like a build process isn't it? Yeah. That's turning our stuff that we've designed into something we deploy, meaning this stuff here on the right. This is an artifact. This is the build artifact that we got from during our build process. However, poorly defined and as you can see it was nowhere in there to store.
So our artifacts in a specially 9 branch and I'll get repo. I didn't set that up that's kind of how it does it automatically.
U SQL script that I had to do my TR? Where is it's not in the replacement.
What about the other dependencies I had like my SQL server database on my storage buckets or my data Lake account?
Where are they well they're not stored in this refund service integration is kind of useful?
Does anyone have a job where they go to the office every day and only work on data factory?
And you know that external stuff no SQL queries right because you don't still those here right so this is useful for that right now.
But any of those other transformations almost all the transformations require some kind of external file right to tell you what to do when there's a store procedure or use SQL script. But those files aren't stored here in the reef is really this is only useful if you only work on data factory and you only use it to copy data in which case you know you could've used FTP.
So what could we do what we could.
Create another repo to put their stuff in.
But now we got to refocus so never got a new problem, so if we have data factor in one repo everything else. The other repo. Somehow, we have to manage the dependency between those 3 points, but they have to be code version 'cause you know all these things are quite tightly, coupled together.
So really we've created a new problem of having to synchronize 2 get repose.
Because of this simplification of get for data factory, there isn't a correct right.
This never ends well have been a bunch of attention. This already is to create these abstraction, so it gets too hard. Just make it easy by adding this interface, but actually those get commands like margin branch. These are already abstractions for underlying more complicated things.
Assaulted these tools always end up I think being a simplification too hard and make it kind of easy to paint yourself into a corner.
So.
Is the portal a dead end?
Well.
2 or 3 years ago when I was doing this right customer wants to use data factoring going to look at some videos on channel 9 about this.
So I look at the outlook at this video of someone from Microsoft doing this demonstration using the old port or not this one. There's lots of like copying and pasting from now get things from OneNote to another lesson. I thought you know what? What I learned from watching those channel 9 videos about data factory is that basically no one was using data for otherwise they would they would have to solve some of those like.
You want us to be good 'cause I think the infra in browser development doesn't it.
I think you know a divorce, a quite a long way down this road with cloud 9. Git lab quite a long way down. This road with web ID and get labs web ID is actually embedded tick embedded the same editor component.
As Visual Studio Code, Microsoft have some support for this in their portal as well right. But it's not really a thing. I don't think in the way that it is for some developers that enables us to take the.
Developers laptop out of the development equation right so with windows updates, not working right and Resharper. Breaking your keybindings in Visual Studio and all that stuff goes away doesn't. If you can develop in the browser. However, this portal isn't there yet.
How did that come about? I think I know how this came about right so?
Imagine someone say if I were company and he says, well, we've had our enterprise devops consultants in.
And I say that buy from now on has to have devops included in the camp right. So then they got from Microsoft says Well, you know that there was 2 when the customer says yes. We want to get an enterprise data orchestration framework and we think about using Luigi because it was.
It's an tribes already built it but now under the microphone is well. Yes, I know. Microsoft loves open source. But we didn't mean that open source. We met the other open source. We've got this thing called data factory stration at all. We need to do is Rob develops on it, and it will have devops in the camp.
So this is a Bolt on to take a box with some custom.
That says, we're not going to use data factory and as it has devops so it's like right great.
Hey, you got.
But we actually do if we want to work and I can knows well the answers here in what we deployed right so this is the published bron report when it contains an arm template would not talk about it much. But we can say this is adjacent definition of everything we've got another top here. This is a data factory here. Some of the teams are in it. That's a reference to a link service for data Lake.
Resource manager Let's create templates that define the infrastructure and everything that goes around it. And this is the best part here. We can repeatedly deploy our solution throughout his life cycle and have confidence ever pose a diploid resource is are deployed in a consistent state was great right. This is what we want to do.
I think at the moment where it's not with anything. These templates through Visual Studio code. If anyone here is here, like on company expenses. There's an Indian restaurant pretty fancy next door dash is right, and if any of you are familiar with it. So Ashes has got yeah, you've been to actually. Yes, yes, Ashley's got 4.3 stars. Ryan this is one half stars.
Sorry. If you work for big customer and all these vendors out here says you tomorrow. Hey, how about lunch Ashes because I know I'm going to lunch of ash is going to stay here, and edit my own templates in Visual Studio code because it is .2 better than Ashes.
And what it does do is look at all the resource is in our resource group in one place right so not only the data factory, but you know the link services to go with the pipeline with database server. The storage account. Data Lake account. All this can be defined in one place in one repo and it looks like this right.
This is both like storm across again some parameters right so we're still parameterising but we're parameterising I will see so we can create a data Lake store with a certain name and sample all these other properties.
Here's where we define our parameters, but for every parameter.
Some things now, she will have to have a unique name because they're exploding through public endpoints. I can never remember which they are, which they aren't. But like storage is one that is. I think the database thing is so that means. You can't username for your storage account that someone else is used already we got this function in Azure.
I want to cry. I string scope to this resource group, so that we know we're going to unique ninth alright.
Price in our resource group ID and we're going to fault all of these parameters.
Pull up a server names and they service names to use these unique strings are there are some things we can do as well right so we got an Azure data. Warehouse we might say for development. We want small cheap one. DW 100 for production. We want another larger size and all these things going to be parameterized.
What happens when we deploy it though?
As we got Resource Group. It looks like this we got recently names, but actually we don't care about this. This is created a unique name that we know we're going to have anything else with some confidence based on our resource group services.
Environment we can use private to ourselves drawer development right, but how do we get?
Our templates from our desktop Visual Studio Code 2 Microsoft Azure.
Yeah.
I see most people that tried to avoid using the D word in my talks at all.
Like marks off of work around this by putting it in the name of the product. The other consequence of this is that ice with this tool no longer fits in the box on secrets website does Azure. Devops is longer than VSTS right right.
And our naming aside.
What is Azure Devops all about well it's a bunch of these things which are more or less familiar under different names. Important ones to die. While Azure Board member that is a kanban pipelines and Repose Repose R get repost or possibly the other one pipe.
This build definition, so this is familiar from before I know that if you do this now, you get the yamil code that is there by default, but this looks a bit better on the screen right.
So what we doing bill from bill. This project what we're going to do a deployment to Azure going to deploy our resource manager template through a resource group.
But it's gonna be a special resource group.
We're going to pass the name of our branch in the repo our feature branch.
3 even numbers all script was not going to get it going to get.
A resource group in Microsoft Azure. That's named after my branch in source control.
And it means we're going to get resource is that a private just for that branch. When are we going to do this build going to overtime so every time we push to commit to a branch? We're going to update our infrastructure in Asheville to match what we got in source control and we're going to do that. For every single branch every time we make a change. Obviously, the first time is going to be slower because.
Infrastructure especially if you're creating something slow to create like a data warehouse. But the subsequent runs faster 'cause. It's not instant cells to check what's there, but faster it's not creating big resources.
So we created branch.
This is in as your repose.
We create a feature branch.
And so that's just a copy of Master.
But we gotta build and you can see, there, possibly that little blue thing is going around right.
And here's the job building.
But it's created in Azure.
A dedicated environment just for this approach.
So how's that right? We can do our like private testing and development. Just on this resource group here without interfering with any of the other developers.
Am or not, I'm going to massively skate over testing 'cause a time regarding tests. We can run. Those tests so we don't get that thing, which is really common, especially in shared environments would like Oh, yeah, well right. This change, now nothing works right back. So I don't know if it was working before I started making change because we can run our tests like overtime whenever.
We do a deployment.
We can ignore that when we start we start from a good place.
The dice factor that we created through this behind the same script right. We can't actually open it and debug it here in the portal what we can't do is use the source control integration. From here in the portal right right because that requires a special set up in the repo. But we can still do everything we want to do in data factory as long as we get it back into those codes.
So when we complete the feature.
With the approval quest going to take this off some stuff here create a comment.
And here we are about to complete the merge.
And crucially were going to delete the feature branch offices.
I don't know why people don't do it right people don't. They they merge feature branches into master and whatever develop branch right then always always leave the feature branch there anything? Why is there even an option to like not delete the features right you don't care about it now it's much, anyway, going to merge that feature branch into master and delete the branch.
Right well, we still created that we created that resource group that matches this branch would be using just for this testing.
And so you can see it quite quickly. That's going to expensive right. If we're going to create a new branch special fiscal real expensive stuff in it data warehouse right premium storage this stuff. You don't want to leave those hanging around in your other account when you finish with the feature.
What we can do is?
Use this feature, Azure Repros worse in most get servers to get the event and take some appropriate action. So service works allow us to inscribe your favorite services. When notifying them when events happen in your project. This is events in source code source control.
So everyone's favorite services, obviously slack right so.
Where you work at like this trip is like a slack channel that gets like 10000 updates an hour because anytime anyone on the planet like Press, a key on their button on their keyboard. You'd like to update now so if you want to set that up invest in Azure Repose. This is where you go to set up that slack integration that we're not going.
Service.
Going to service hook, I'm going to say right.
Anytime we going to merge full request.
And it successfully merges into master.
We're going to far in event. Yeah, you could enter slack. But in this case, we're going to send it to this URL here.
And this URL you can see I think it's going to highlight.
Is in Azure automation? So when we might approve request? We're going to send this event to this web block here in Azure automation, which is going to do something useful.
What we're going to send this to Pedro is pretty long. I just copied out this place we can see all this information about the?
Action that happened in source control app developer. It was what time it was what date it was.
And also we can see.
But not there.
What branch was being merged into master we get when I'm in a branch so that in Azure automation, we create that URL.
And Hook, it up to this script here.
So here we're saying whenever I receive that message from Azure Repose look for Jason.
And find.
The name of the branch that we've just merged into Master.
Then go and look in the same Azure account.
For a resource group that's named after that branch.
On the way to resolve script.
So that means.
Then we merge the PO request and complete it.
I have a process was it's a cherry pit and why.
What would cherry pick be the first thing you would do after merging feature branch into master?
I think there is a reason, but that's for another talk, but anyway, we merge the pull request. Some some other stuff happens right.
It's got triggered us run book in Azure automation, you see it says it's running.
I was going to destroy that Resource Group and clear up all the resources that we've used.
But something else, something else happened right because of that continuous integration trigger. We've also created a commission master the merge commit right where we put our feature branch into master so we get a new build of Master and actually that's going to build and deploy a resource group that's named after the master branch called Master and this here.
You can kind of think of as being this is kind of the integration environment, it right. This is where we deploy master every time we update it to make sure it's all still available at all still working.
And the our tests are still passing in that Master Branch.
The final thing we do is we save some stuff out of the build so we deployed this thing to master I mean, I was alright test surpassing.
But here we say, Alright, so only in this master branch and only if everything is working save that script. We ran to create those resources and that data factory in that way.
Into Azure pipelines, so we can use it again and we're going to use it again.
We're going to use again.
Creating release so this is also called Azure pipelines. I think that right.
This is quite a simple over here on the left that little symbol. There says Yeah. We're collecting our artifacts from loan location and straightaway going to a deployment environment called staging.
At some point after that everything is cool. We're going to do a deployment of this thing called production and that's got some other pre deployment conditions that have to be met will talk about it.
So when we do release a staging.
It looks a bit like what we're doing in those in those build steps earlier with the feature branches. Yeah, we gotta take all those templates and deploy them to our Azure account. But this time, we're not going to create a new resource group with their dynamic made up name that matches the branch. We can use a well known name right. This is my staging environment permanent overtime, so over him in Nova Branch.
Use this resource group, it, I'm down here. The template parameters.
I can pass them in so these resource is that exist overtime. I staging environment. Survivors say generation. You data fax River new name generator new storage account revenue name. So no this is my permanent environment. So I'm going to tell you.
What parameters to use and in fact I can do things like I've not done this once I know I can do things like change the size of the resource is change the cost basically right so I can use a bigger data warehouse or faster storage account for my permanent environments that I had from my federal unless you wanna do the other way around right like.
If you got environment going to keep it around for very long. He wanna make that one the expensive one this one less around overtime might not want cheaper.
In production, we got some stuff we can say, Oh, Yeah, like Mr. Important has to do in approval before we can do a release and then there's another thing called Gates, which I think is new that is to incorporate existing release management versus you have light so if you have a release process that says Oh Yeah, before you can do release Mr. Important has to approve it.
You have a closed change request ticket in like remedy or the other one service now that's where you can do. That kind of integration. So you can actually automate. All that stuff as long as you can like query and get that information into the Hooker.
That was that was quite like a effort intensive thing to do before I think integrating.
And finally we can do our Lisa production. So here again. We supply the parameters were supplying a bigger database datawarehouse tear name and.
There's a real values to our permanent resource group that we use for production so.
In summary this is how well that worked right we create a new feature in its own branch. We want to build within that creates an environment for the feature. We do our testing when we merge the branch. We destroy that environment and we keep the artifacts and we use that to promote to our opponent environments.
Right now, the fun part.
So everyone's going to have phones handy.
Right.
What happened?
Right. Sorry.
Something went wrong.
If you go to mente.com.
All right this second.
Right good.
So this here is do we understand what this tool is doing alright alright?
Quite a few football.
No right OK great right.
Alright good next slide.
So share development environments. How much do we like using shared development environments like you know on Prem or in the cloud.
Like I know who it is it just type that you know.
Yeah, I believe the party starts at 7 right.
Creating an environment for each branch using that thing we just did is a great idea without terrible idea.
20,000 votes for Terra woman.
There we go so what's so great about it.
Isolation yet that's good.
Play in automated yet, all things expect right no surprises.
Yes.
Alright sounds expensive right you should've save that for this slide. This is why it's a terrible idea.
So we can say expensive now where is that bar expensive right so yeah, it is expensive. We're creating a hole in your environment for each for each feature then we got each feature branch. We got open right, but actually this is rewarding good behavior right not leaving features on his open for agents right if you feature. Branches yeah, only stay open for a day or 2 well. That's fine. We destroy the environment. The end right? That's what I was talking about this a while ago.
So, in my imaginary friends from the Internet right and I said exactly this Oh, yeah, sounds really expensive because we feel like 100 open feature branches something about what's wrong with you? Why do you have 100 open feature branches. There are 2 there are 2 reasons for having 100 open feature branches right even like your huge right or you're really disorganized.
But actually the only other expenses that scale like this linearly with feature branches right so for each of these developers right. You have like an aeron chair for him to sit in and write code don't you right. I think that grinder time and that's an expense also that you need one of these rich concurrent feature branch. You working tonight right there. Lots of other things that just.
Cost of having a developer sitting there, writing code have expensive coffee machine and a private resource group in Azure Todo todo testing.
Yes.
That's true, yeah, Summers also cheaper than others right. I think the real killer is like the.
The database ones right right, but with this being a database conference. Those are all the ones that are really expensive in particular data warehouse it's like.
Derivative it disciplines required, yeah right? Yeah, the swear filter is on well done right.
And kabob Easter expensive power isn't yeah, someone branch called the same as production was also but you probably could inhibit that in a free whatever hook couldn't.
Accidentally deleting the Master branch, yeah right.
Will accidentally deleting the Master Resource Group could happen under well. I mean, the whole thing is that accidents happen by people log into the portal and clicking stuff right and this is kind of a way of like skirting all of that isn't it?
So that's about it, I think I'm just about finished they'll take anymore. Questions in the couple of minutes. We have you can either type them in or shout them out. This is supposed to be like some people are funny about asking questions or conference talks are so.
Any questions.
So.
Yeah, so the question is how do you handle shipping data that you require for test so if you're doing a data factory loads for a file? Where do you get the file from that file has to be source controlled with the tests that depend on it, I think yeah alright.
And those tests should probably be in the repo with the source code and there's going to be a problem with that file has to be.
Very large, but feel like functional testing.
If need a terabyte of data to figure out whether you're like, you know, separating the kabobs correctly by type of source right there probably something wrong with your testing methodology. So I think that thing with that is keeping test data small so but you can't go version, it with the tests on the code that you're testing.
This is integration testing there's no point being pretty about the difference between the unit and integration tests were doing these past things you are touching external dependencies by their own box here. You've got the real dependencies. So yes, you have all the things that about about integration testing like slow setup copying files around and all that right, but that kind of comes with the territory.
Anymore, yeah, I think the repo is private at the moment but.
On my website there is a link to a list of over talks. I've done and summertime in particular, the more recent ones have a link to the slide deck. So I know there isn't one for this talk right now, Yeah, but there will be other weekend or something.
Well, you can build your own like kebab based startup you.
I think that you have to be like a super celebrity to have public repose and that's a previous feature is not.
Alright anymore.
Yes, sorry about that.
Yep.
Yeah.
Yeah, there is right, you can sorry.
Yeah, so the question is right, if you have let's say I get Florey type setup where you have multiple long lived branches like developer release or whatever. How do you stop those lonely appliances from being deleted and the answer is?
You have to add more logic to the Azure Automation job. I want to pull off top my head so we have some logic based on like?
Find me the resource group that matches this branch and delete it right.
Jason payload from VSTS from Devops right, but you can get the branch names so you could say.
I think you can probably filter branch me, setting up a hook as well. Actually, yeah, yeah right.
I think disaster wanted cereal trying to switch back to the slides, but I think possibly you can filter by branch at a point in a fire in the hook and definitely you can fill up anything you want, but you're writing code in the.
And the automation run book itself, yeah, so yeah, and you know doing those things more complex but doing gifts more complex. That's like. Yeah, that's the that's the price isn't it?
Yeah, in her car, possibly the pointer far in the hook. I think, but I'm not gonna go back anymore.