Remember my blog post on “Multi-root workspaces in VSCode for AL Development“? If not – it might be interesting to read first, because this is in fact an “extension” (get it?) on it .. so you might say this blogpost “depends” on that one ;-).
I concluded the post with some scripts – and that’s the part I’d like to extend a bit – because I needed more functionality since latest blogpost. And I’d like to refer to these scripts again – and tell you a bit more on how they could make your life a bit easier as well .. .
Branching
The main part that I extended is the “Git”-part. And as an example, I’d want to take you to this blog post from Michael Megel. Michael talks about the “Release flow” – and how much it makes sense for ISV partners in Business Central. Well .. now take into account the multitude of apps that we might have. In our company, we have 22 apps at this moment (x2 if you count the corresponding test-apps as well).
In terms of Git/DevOps, that’s 22 repositories. In terms of VSCode, that’s 44 workspaces. In terms of release management: that’s branching 22 repositories the very same: if I need to create a release branch, I need to create the same release branch over all repositories at the same time.
So indeed – we choose to NOT have a release per app – but rather have one release for all apps – so when we create a release branch – we basically have to create the same branch-name in all apps. Fairly simple – but if you have to do that manually for 22 repos – that’s going to be tedious (and any repetitive job that is executed manually, just cries for mistakes).
This is just one example of many that needs to be solved when you have multiple apps/workspaces/.. .
Scripting
Indeed – that’s where scripting comes into place. And having a multi-root workspace in combination with some PowerShell Scripts, just makes your life a bit easier. Do notice though that I’m not making my scripts part of the multi root workspace. That just doesn’t work – the Al Language Extension does “something” with the F5 key that kind of like disables the ability to run any PowerShell script that’s part of the same Multi Root environment (F8 does work – but I need F5 ;-)). So – I always have a second VSCode open with my PS Scripts.
The scripts
The scripts are still located in the same place as mentioned in my previous blog, but I have more now. So you have more now as well ;-). Do notice it’s based on my PowerShell modules. So I do advice you to install them, or not everything might work (thank you for the download count ;-)). So – this is what I have today:
Make sure all variables are correct in this file – otherwise all below scripts won’t work correctly. As you can see in this script, it will not look at any workspace-file, but it find all app.json-files and treat all directories as “targets” for where to execute the below scripts. I might change that behaviour in the future though – I don’t know yet – this works for me now ;-).
This script will compile all apps in your Multi Root Workspace, in the right order (it will use the scripts I blogged about here to determine the order). And then it will call the “Compile-ALApp” that’s part of my module “Cloud.Ready.Software.NAV“, which will use the alc.exe in your user profile (basically from the AL Language extension in VSCode) to compile the app. I don’t use this script too of the though – only when I really need to for example have all updated translations files.
Well – if you use one Dev environment, for all your apps – it’s good to have one vscode-file for all workspace. I know there is a way to update your workspace file with configurations – but I personally don’t use that. At least this way, I’m still able to easily just open the individual workspace, and still use the right launch.json.
This is a strange one. This script is actually going to copy a command to my clipboard that I can execute in the terminal of my MultiRoot workspace. It will open all App.json files. Few reasons would be interesting:
- you would like to manually open all manifests to do a similar change to all of them
- You would like to just open a file of all workspaces to start the AL Compile and find all code analysis problems in one go (an app is only compiled when one of its files are opened)
This script will remove all symbol files from all workspaces. Especially when you change localization or version, this is really useful.
Yep – a loop for downloading all symbols for all workpaces. It isn’t very useful though because in a fresh environment, it will most likely not be able to download all symbols – although, when I start a very clean environment, I usually cleanup all symbols and run this once – then at least I have “most” of them. It doesn’t hurt to run this in the background ;-).
Simple loop to put all translation-files in one zip-file. Easy to mail to your translator.
Git_CreateBranchFromMaster.ps1
This is where the branching comes in play. This script will first synchronize master, and then start a new branch, all with the same name, for all your workpaces. Especially interesting to synch branch-naming for multiple repositories (like necessary in this Release Flow as mentioned earlier).
Just imagine, you messed up – and you don’t want to do some kind of edit in all your workspaces, and want to execute a “Discard All” on all your workspaces. That’s exactly what this script does.
If you don’t want to discard, but edited files in all your workspaces, and you want the same commit message for all your repositories: just a script that does loop through your repos, and will stage and commit with that same message.
This doesn’t need too much explanation: it will switch the branch to another branch.
Again – exactly like it says – it will update the master-branch for all your workspaces.
Git_UpdateBranchFromMaster.ps1
This one will update a branch from master branch. It will first sync the master branch (make sure it’s up-to-date with the remote), and then update the selected branch with master. This might result in conflicts, which you have time to solve in VSCode.
So, on which workflows do you use it?
They are very useful in many scenarios .. . Let me explain a few of them.. .
Translations
We try to translate in batches (no – developers are no translators. Development languages or not “normal” languages, you know ;-)). As such: send a bunch of translation files to the one that will translate. When I get that back, I will import the files, and commit. The workflow is something like:
- Create a branch for all workspaces (Script: Git_CreateBranchFromMaster)
- Usually called “Translation”
- Compile all to create an updated .g.xlf file (Script: Apps_CompileAll)
- Use VSCode “XLIFFSync” extension to update all translated files
- Commit “Before Translation” to git (Script: Git_StageAndCommit)
- Create a zip-file (Script: Apps_ZipTranslation)
- Import translated files to the right folders (manually)
- Commit “After Translation” to git (Script: Git_StageAndCommit)
- Pullrequest all changes to master (Manually in DevOps – Intentionally – I want PRs to be manual at all times (for now))
Release Flow
As mentioned above – there are many scenarios where you would like to “sync branch names” across multiple repositories, like creating the same release-branch for multiple repositories in an AL Dependency Architecture.
Simple:
- Create a branch for all workspaces (Script: Git_CreateBranchFromMaster)
- Create/modify the pipelines (usually yml files across all repos)
- Commit to git (Script: Git_StageAndCommit)
- Secure you branch in DevOps (Branch Policies)
Major upgrades
Microsoft comes with new major release twice a year. And for major upgrades, it usually takes some time to prepare your stuff for the next upgrade. We simply follow this workflow:
- Create a branch for all workspaces (Script: Git_CreateBranchFromMaster)
- Fix a particular problem
- Commit to git (Script: Git_StageAndCommit)
- Pullrequest to master when done (manually in DevOps)
And probably steps 2 and 3 has to be repeated a few times – and that’s again where the scripts become very useful ;-).
Conclusion
If you are a heavy user of multi root workspaces in AL Development – give these scripts a spin. I encourage you ;-).
14 comments
Skip to comment form
Okay. You’re not really doing this right (IMO 🙂 )
Firstly, a terminology nit-pick, Git branch names are ephemeral, they have meaning only in the repository they are defined. They (nearly) always get renamed when transferred between repositories eg: master -> origin/master. So the phrase “Creating the same branch in a different repository” usually means literally creating the same sequence of commits in another repository as that is what the “branch” is, not it’s name. Often it’s close enough, but not in this case.
Secondly, you really should be using git submodules which are designed for this use-case where you have commits in multiple repositories that are linked together commit by commit.
Personally, I dislike submodules and tend to reimplement the basic functionality using worktrees but this still means I do NOT have multiple release branches. Just one in the main branch that links to the actual commit ids that constitute the release. Only if there are concurrent changes do you get ‘release candidate’ branching in the subproject (and even then this is a reasonable time to rebase the dev branch on top of the release candidate).
> (an app is only compiled when one of its files are opened)
Not your fault, but: “stupid, Stupid, STUPID!!! … Idiots!”
This is why I think AL extension’s “support” of multiple projects is “not good”, for example, this “wonderful” feature makes “find references” very flaky.
Anyway, nice pair of posts.
Author
You are probably right – but at least it works for us ;-).
I admit – git submodules are new to me. I don’t know how that works, and I don’t know how that would look like in DevOps, including Build/Release pipelines.. . I would love to have a chat on that though .. would be really interesting!
Author
I extended on my approach a bit further down the comments – may be that makes more sense..
You could also look into maintaining your extensions/modules in a monorepo.
Since AL is scoped from a folder level, you could structure a single repo and generate .code-workspace files for the levels you want.
Root
– – AL
– – – – Extensions
– – – – – – ExtensionNameA
– – – – – – – – ModuleNameA
– – – – – – – – ModuleNameB
– – – – – – ExtensionNameB
– – – – – – – – ModuleNameA
– – – – – – – – ModuleNameB
– – – – Extensions_Tests
– – – – – – ExtensionNameA_Tests
– – – – – – – – ModuleNameA_Tests
– – – – – – – – ModuleNameB_Tests
– – – – – – ExtensionNameB_Tests
– – – – – – – – ModuleNameA_Tests
– – – – – – – – ModuleNameB_Tests
You’d a have a .code-workspace file generated from all .app files in AL, Extensions, Extensions_Tests, ExtensionNameA, etc.
Therefor you could publish individual Modules, Extensions, with or without tests, while also allowing for gitignored prefix like Custom_ (for example Custom_ExtensionNameA) for experimentation and moving things around/splitting your apps.
Microsoft e.g. uses this for their “AL modules” development in the System Application.
I think having separate git repos for projects that share history can be cumbersome (and git submodules can be painful to work with). You still have seperation of concern via folder structure instead of repositories, but I think it’s worth it.
Author
Thanks for you input.
We did look into that. But we encountered many reasons NOT to do that. One example: ANY PullRequest is going to be immensely slow – having to compile/publish/test all apps .. that was a no-go from quite the start.. .
Thing is – a lot of the apps are used independently. There are just a others that have quite some dependencies as well – so I kind of have to release all together, but still having them “forced” independent as well – I don’t know if that makes sense at all.. .
Honestly – I wouldn’t know why to change this – it works pretty sweet as it is now…
Yeah, I would’ve thought the git diff could be used to only compile/publish/test “dirty” folders but this requires additional scripting too.
I guess these could be approached both ways (mono vs multi repo) and since you’ve already got the scripts working for your setup I see nothing wrong with it and I’ll definitely try it out.
I just thought the hierarchical idea was flexible and predictable and not having to think about repo paths and syncing changes/relase branches between repos sounded great, but there are pros and cons with both methods.
A common example of state mismatch happens when A is a repo for an extension and A_Tests is another repo for the test extension of A. A depends on A_Tests to pass in build pipeline but A_Tests depends on A as an app dependency.
This gets really annoying since you need matching branches for both of them, map them together in the build pipeline and make sure they’ve both been pushed before trying to build/test them.
Do you have a specific solution for that case? I would be interested to know how it works out for you and might have missed it if you’ve mentioned it in a blog post before. 🙂
Author
The tests are always part of the same repo – because we implement ATDD. No Tests, no code being approved ;-). On repo is one app, but one app is actually always two apps: the app and the test app. Here is an example: https://dev.azure.com/ALOpsWebinars/03_SaaS%20Deployment/_git/BASE
Hi Waldo,
sorry but that sounds strange to me. Why has every repo be on the same “Branchname”? and why I have to branch every App when I only want to add a new field one app. Or did I get something wrong?
Yes, it is very handy to have all Apps in one workspace but this feels to bend it to the maximum.
We started to implement nuget. Every App has his own Project/Repository und release pipeline to publish a nuget feed. With this feed the other app could download the corresponding version of this depended app.
What did you think about that solution?
Sure, not all apps in a multiroot workspace but if you want, you could add all repos in one workspace file.
Author
I understand it sounds strange – but look at the apps being dependent from each other. If you do development – it happens a lot (not all the time, but a lot) that you basically develop in multiple apps at the same time. Releasing them together (as the same “release number”) makes sense in that case.
If you talk about apps that are all completely independent .. this obviously doesn’t make sense.. .
When we release the product (which means: releasing multiple apps that kind of like have to work together), we release all of them at the same time (release flow). Having the same names for all of them makes it really easy to talk about “AppX, release 5.1”, or just “release 5.1 of the product (which is the collection of apps)”. Customers are implemented with a subset of that collection of apps. We never do versioning “per app” (versioning is done completely automatic and driven by DevOps), not having to remember to manage specific version switching on (one or more) dependent apps, just because you did change one app, which means now all your others need to be dependent from a specific version of another – if you think about it – that is a highly complicated thing to do, in a architecture with many apps, and a lot of interdependencies. You completely rule that out in my case: dev branch(es) are all interdependent, release-branches as well (dependencies between apps with same version are guaranteed) – that’s it. Nuget does not solve that for me in my opinion .. .
And yes, I could also put all of them in one repo. But I don’t want to :-). There are quite some reasons. Apps should not be *that* interdependent. Developers should not add dependencies (this needs to be controlled on DevOps level – avoid as many dependencies as possible – will make apps more independent (obviously)), if an app works in an interdependent world, but hasn’t got any dependencies itself – it should work independently, and should be built independently, … . So – the way we set it up (separate repos) speeds up our builds, and give us more flexibility and control for the app itself. It’s also easier to control build pipelines, control dependencies from DevOps, code review, split responsibilities (which you want to do with a huge set of apps), … . Also easier to implement unit testing (per app) and integration testing (yet another app that is built and tested with all apps installed), and set up pipelines accordingly.
Trust me – I understand it sounds strange. And it’s also “just” when you have product with a multitude of apps that are “as independent as at all possible, but actually also released and implemented as one product”. May be I should have been clearer about that.. . This thing has gone through quite some changes and reconsiderations (over a period of 2 years) – and we ended up that this works best for us.
I’d love to chit chat around that when we have the chance though :-).
First, one thing I forgot: thanks for your post!
I realy enjoy everything that we as partners could share 🙂
To the reply 🙂
Short: All of these apps will have the same release and the same version no. Correct? 🙂
Okay. Within the case that there is one app that has some smaller depended apps (gears of the bigger one like Base App = Financials, Purchase, Sales ….), I could understand the choice of the same version no. and release branch.
It would realy not making sense that these internal (but depended) other apps have another version.
I thought about e.G. an extension of an app.
The starting app is full usable without the new extension app. The extension app is not working without the starting app.
In that case, I think that the apps will have diffrent version nos. and releases.
Why should I update the starting app while I was working on the extension app. Sure, If I had (like you said) work on both, both got an release. With nuget those versions are connected.
Or an “Core App” in which some basic stuff is implemented. This app is in some different apps which are not depended to each other but all are dependeded to that “Core App”.
In that case, all apps have diffrent releases and versions and if we update the “Core App” we have to update the corresponding other apps too (if we want that feature/bugfix there). Or (with nuget) we will load the old artifact and we could not use the new implemented features.
What would you think about thise scenarios? 🙂
Our scenario will take time…
If you change the Core, some pipelines will fire and also the “Push To Nuget Feed”. After that, the other apps could consume it.
And also its not fully proved. We are in our discovery phase if this could be a possible, good usable way 🙂
But in the next thinkable step, we want to load (e.G.) your restApp. If I could download it via a nuget feed I could publish it to my containers or also to the database of a customer as “part” (not realy but necessary app) of my App. Could be realy usefull. a PSGallery of Apps.
Would you choose for those kind of apps the same structure as you write above?
But we realy did not count with your dimensions 🙂
(after writing that wall of text I could say: sorry :D)
Hmm, Waldo, I have mentioned submodules already two years ago, seems you already forgot… 😉
We are using submodules and multiroot workspaces (MainApp, TestApp, Dependencies from submodules…) – yes, adding dependency means extend the repo with new submodule and add the correct folder into the workspace. But it helps me to work with multiple apps in one workspace, still having separate repos and separate pipelines. I am not solving versioning, because major and minor version is driven by developer (if change in interface than increase etc.) and because backward compatibility, you do not need to rise minimal version of the dependency if the used version is enough for your code.
All works. Seems I should find time to write something around or make a webcast… 🙂
Author
“mentioned” .. man, I need so much more than “mentioning” before I get that ;-).
Well .. I guess there multiple ways to go to Rome. I will look into this .. but I can’t imagine a repo (submodules or not) with 44 apps at this point.
Hi Eric!
a small comment to your F5/F8 AL/PS problem…I agree. it sucks….but since I do not know your scripts I can tell you what we did to bypass this: Utilizing “Tasks” in VSC. They can run Powershell! You can influence the input!
Like this we have e.g. a task to spin up our development container. Or restart the Servicetiers in a container or do other funny things. That makes it for newbies quite easy to get started. maybe something you can use as well.
It must not be always a full extension for VSC to get stuff done…
A bit the same but not as practical: Command Palette > Powershell: Open examples Folder -> ExtensionsExamples.ps1 -> read the header and try it out 🙂
You can execute a sequence of tasks too, but AL commands are configured in a way so they can’t be executed from tasks (or launch.json) without throwing an error – which stops any following tasks from executing.
I’ve submitted an issue to the AL repo and also to the VS Code repo to fix the issue in general.
https://github.com/microsoft/AL/issues/5859
You can then overwrite the F5 keybinding and insert your own scripts there:
https://github.com/microsoft/AL/issues/5262#issuecomment-575799815 but what I proposed to Microsoft would make us able to execute a mixture of AL extension commands and the Powershell extension commands, e.g. depending on which workspace we’re using, all without a custom extension. Until this gets fixed you could in theory just add a task to a custom VSCode extension which just executes a ${command:} but swallows the return type error.