Tag Archives: Build Scripts

Understanding the Build-AXModel function

08 Thursday Oct 2015

Posted by in Powershell

≈ Leave a comment

Tags

, , , , , ,

I’m going to start off with how I set up TFS for a build.  I have my main code branch in TFS, D2D_AX_DEV, that is used for all of my development environments.  Each AX development environment has TFS setup and pointed to this code branch.  I have also set up a branch off of D2D_AX_DEV called D2D_AX_REL.  This is my release branch.  I merge everything into the release branch that is going to be included in the build.  I then point my build function to the release branch.

The Build-AXModel function is broken up into 5 sections:

  1. Outer section at the beginning and end that set up the variables passed through the parameters and that email start and completion of the function.
  2. Set up build variables: This section uses variables in the Build-AXModel_Variables.ps1 file that is passed in through the VariablePath parameter to set up the environment for the build and model export.
  3. Prep build environment:  This section restores the AX environment back to a vanilla instance for the build and gets TFS and the combined .xpo file ready.
  4. Import/Compile AX (Main build step): This section pushes the TFS code into the build environment and compiles/syncs everything and gets the environment ready for a model export.
  5. Export model: This section exports the model file into your build folders.

The individual steps and explanations for what they are doing can be found in my earlier posts.  I have done some things in my Build-AXModel_Variables.ps1 file to make this more of a generic process to be re-used however and I would like to clarify what I’m doing.

There are multiple places in my build function where I loop to allow functions to work on different layers and models (model creation, .xpo creation/import, label file import, VS Project import and model export).  In my original process this wasn’t necessary as I only build one model in one layer.  The way it is currently built however, allows for multiple layers and models.

I handle this looping process using foreach statements and ArrayLists in my variables.  I used ArrayList vs array as the behavior with ArrayLists was a lot more dependable when looping.  It is a simple construct.  I build variables for the individual parameters needed for a function call.  I then build an ArrayList to hold grouped variables for a function call that will allow me to loop through an ArrayList of my grouped ArrayLists and call the function multiple times.  My loops are only through the VAR layer and for my D2D model but you can easily add more variables for layers and models to these ArrayLists to allow multiple function calls.  Make sure you build your ArrayLists in the order in which you want to process the layers and models.

ArrayListEx

The area that is most likely to need customization to fit your specific build are the steps in the “Import/Compile AX” region.  You may need to change the order of the server/node compiles or change the server compile to a client compile.  You may also need to change the function used to import the VS Projects and add some node compiles if you have some complicated dependencies.

As I said in the blog post covering the function, I’m not planning on putting this function into the module because of the likelihood that you will need to modify it to fit your needs.  I’m hoping that some users will be able to use it out of the box but I’m sure there are some who will need to make changes.  The steps in my version of the function fit the current needs of my build process.

Build-AXModel custom PowerShell function

23 Wednesday Sep 2015

Posted by in Powershell

≈ Leave a comment

Tags

, , , , , , , , , , , , , , ,

I apologize for the extra week between posts.  It has been a lot of work to get this function ready to post.  I built the D2DDynamics PowerShell module using cleaned up functions that were based off of the original functions I used to create my build.  Even though I tested all of the functions as I built them, my actual build was still using my old functions.  Getting my new build ready required me to bring my module into my build environment and re-write and re-test my build.  I also wanted to get at least one build done in my environment using the new function before posting it.  Hence, the extra week.

I’m not going to get too in depth regarding the function this week.  I am making it available so people can look at it and try and get an idea of what I’m doing.  I will be covering the specifics of it in future posts.  Also, I will not be including it in the module.  It is possible that a large percentage of users will be able to use the function as-is to create a build but some users will need to modify it to make it work for them and all users will need to modify the variables.  You should add the function to your session using your profile.

I have also updated the D2DDynamics module to version 1.0.1.  This release includes some documentation changes, a new Comment parameter for the Sync-AXTFSWorkingFolder function and I’ve exposed the Send-Email function because I use it in the Build-AXModel function that isn’t part of the module.  To use the Build-AXModel function, you will need to download and setup the latest version of the module (1.0.1).

The Build-AXModel custom PowerShell function takes 3 parameters:

  • BuildNumber (The build identifier)
  • ConfigPath (Client configuration for the AX build environment)
  • VariablePath (Path to a file used to default the parameters/variables)

All 3 variables should be used, although you could specify the ConfigPath in the Variable path file if you’d like.  I have defaulted both values in the function that I use already to make my life simpler.  Feel free to do the same thing in the function that you use.  This function is not designed to be used “out-of-the-box”.  You can customize the defaults and you will need to customize the Build-AXModel_Variables file that is used for the VariablePath parameter for this function to work.  I will get into what/why/how to use the variables in a later post.  For now you can look at the 2 files (Function_Build-AXModel.ps1 and Build-AXModel_Variables.ps1) and get an idea for what is going on.

There are 15 basic steps to the build:

  1. Sync TFS and set the label using Sync-AXTFSWorkingFolder
  2. Stop the AOS
  3. Restore the build environment to vanilla using Restore-AXDatabase
  4. Create the new models
  5. Delete the old artifacts using Clean-AXArtifacts
  6. Start the AOS (Build environment is now clean and ready for the new build)
  7. Generate the combined .xpo files using Combine-AXXPO
  8. Import label files using Import-AXLabelFile
  9. Load the combined .xpo files using Import-AXXPO
  10. Compile and load the VS projects using Start-AXMSBuildImport
  11. Compile the AOT using Start-AXBuildCompile and Compile-AXXppNode
  12. Compile CIL using Compile-AXCIL
  13. Sync the data dictionary using Sync-AXDB
  14. Reload the AX artifacts using Clean-AXArtifacts (This step helps with label file issues when pushing the model)
  15. Export the model file

D2DDynamics custom PowerShell module

31 Monday Aug 2015

Posted by in Powershell

≈ Leave a comment

Tags

, , , , , , , , , , , , , , , , , , ,

The D2DDynamics custom PowerShell module is finally available in Codeplex.  The 1.0.0 version includes 15 exportable PowerShell functions and 3 internal helper functions.  They are:

The 3 internal functions are:

This module is dependent on the 0.3.5 release of the DynamicsAXCommunity PowerShell module among others.  There are also some variables that should be set up in your profile if you are planning on using the functions.  The profile that I use can be found in Codeplex at Source Code -> Powershell -> Profile -> Modules -> Microsoft.PowerShell_profile.ps1.  Please use this as a reference for setting up your own profile.

To install the module, copy the D2DDynamics folder to the module folder for PowerShell.  On my Server 2012 server, the location is “C:\Windows\System32\WindowsPowerShell\v1.0\Modules”.  After the module is there, you should be able to open PowerShell and add it with the Import-Module cmdlet.

Future posts will include the steps necessary to create a custom build process using the module as well as more functions that can be used for deploying your build to other environments.  Please let me know what you think of the module and if you find any problems with using the functions in your environments.

Import-AXVSProject custom PowerShell function

18 Tuesday Aug 2015

Posted by in Powershell

≈ 1 Comment

Tags

, , , , , , , , ,

The Import-AXVSProject function is the second of 2 functions to support more complicated Visual Studio project scenarios.  I have taken this approach from here.  I don’t personally use this approach in my build as my current environment doesn’t require it but this function along with my earlier post should allow a user that does have these complicated scenarios to import VS projects for a build.

This function uses a system class to import the VS project.  The AX SysAutoRun functionality allows users to call a static method of a class in AX.  The AX system class SysTreeNodeVSProject has a static method called importProject() that takes an argument of a VS project file and imports it into AX.

Using the Import-AXVSProject function in conjunction with the Compile-AXXppNode function will allow you to import VS projects and compile them in any order that you need to.  If you have projects that are dependent on AX code to compile correctly, you can compile the AOT or specific nodes of the AOT before importing and compiling the project.  If the AX code or another VS project is dependent on the VS project, you can change that order.  These functions allow you to import and compile in a manner that gives you control over order and saves time from a full AOT compile.

The Import-AXVSProject custom PowerShell function takes between 1-8 parameters:

  • ConfigPath (Client configuration for the chosen AX environment)
  • VSProject (The VS project to import)
  • LogFile (The logfile for SysAutoRun.  Has a default value so you only need to specify if you want to change the location.)
  • Timeout (The value defaults to 90 minutes but can be overridden if desired)
  • SMTPServer (SMTP server to use to send the email)
  • MailMsg (Net.Mail.MailMessage object used to send the email)
  • AXVersion (The AX version.  It defaults to 6.)
  • VariablePath (Path to a file used to default the parameters/variables)

This function can be found in Codeplex.  The steps of this function are:

  • Load the variables if a VariablePath parameter is used
  • Get the AX environment info using Get-AXConfig
  • Set up the AutoRun xml command
  • Get the AutoRun xml using the Get-AXAutoRunXML function
  • Call the Start-AXAutoRun function to compile the node
  • Send the success/fail message

This is the last function necessary to create an automated build process.  In the coming weeks I will be cleaning up my current posts to fit with some changes I have made to the existing functions and I will post a blog about putting these functions together to allow you to automate your build.  I will also be releasing a module to Codeplex in the next month that will have all of the functions packaged up for easy deployment.

Compile-AXXppNode

07 Friday Aug 2015

Posted by in Powershell

≈ 4 Comments

Tags

, , , , , , , ,

The Compile-AXXppNode function is the first of 2 functions to support more complicated VS project scenarios.  I have taken this approach from here.  I don’t personally use this approach in my build as my current environment doesn’t require it but this function along with my next post should allow a user that does have these complicated scenarios to import VS projects for a build.

This function can return the compile results in an email but it requires a code change in the SysAutoRun.execCompileApplication() class method.  When compiling everthing, the SysCompileAll::compile() class method calls the SysCompilerOutput::exportLog() class method.  This exports the log to the user’s “C:\Users\..\Microsoft\Dynamics Ax\Log\” folder.  If you want this log to be available, you need to add the SysCompilerOutput::exportLog() class method call to the node compile section of code.  I realize that this is a chicken and egg type thing as this custom code would need to be imported into the build environment before importing and compiling VS projects but it is possible to implement.

The AutoRun xml does have the ability to specify a log file as well but this log file will only show you what nodes were compiled.  There is no message regarding the results of the compile in it.  Because of this limitation, I didn’t used this log in the function.

SysAutoRunCompileLogChange

The Compile-AXXppNode custom PowerShell function takes between 2-7 parameters:

  • ConfigPath (Client configuration for the chosen AX environment)
  • Node (The AX node to compile)
  • Timeout (The value defaults to 90 minutes but can be overridden if desired)
  • SMTPServer (SMTP server to use to send the email)
  • MailMsg (Net.Mail.MailMessage object used to send the email)
  • AXVersion (The AX version.  It defaults to 6.)
  • VariablePath (Path to a file used to default the parameters/variables)

This function can be found in Codeplex.  The steps of this function are:

  • Load the variables if a VariablePath parameter is used
  • Get the AX environment info using Get-AXConfig
  • Get the Compile node AutoRun xml using the Get-AXAutoRunXML function
  • Call the Start-AXAutoRun function to compile the node
  • Send the success/fail message

As I said in the last post, I have been in contact with Martin Drab.  He is developing the DynamicsAXCommunity PowerShell module that I piggyback off of.  He has incorporated the last change that I needed for my functions to work in the latest build of the module (0.3.5).  I haven’t tested it out yet, but my functions should now work with his module with no customizations.

Start-AXMSBuildImport

23 Thursday Jul 2015

Posted by in Powershell

≈ 2 Comments

Tags

, , , , , , , , , , ,

This has been an extremely difficult blog post for me to write.  I have been struggling with it because I realized that I didn’t really understand the process that I was using to accomplish my VS project import.  I took an extra week to research it and hopefully write up something useful and I have actually discovered that my process may not work for a lot of people.

I am using the ImportVSProjects.proj with msbuild.exe that I found in the Dynamics AX Build Scripts.  This project is passed into msbuild.exe with parameters that allow it to loop through a TFS directory and import the found VS projects into Dynamics AX 2012.  I have since discovered that there can be issues with projects that have dependencies on AX objects or other VS projects.  I personally don’t work in an environment that has complex dependencies, so I’ve never run into any of these issues.  There are other solutions that involve using the autorun.xml to call SysTreeNodeVSProject::ImportProject and compile for a specific AOT node.  I will be exploring these in future posts but the current method is the one I use.

The Start-AXMSBuildImport function takes between 1-14 parameters:

  • ConfigPath (Client configuration for the chosen AX environment)
  • MSBuildPath (The folder location of msbuild.exe)
  • ImportVSProjectsFile (The ImportVSProjects.proj file to use)
  • ModelFileFolder (The TFS working directory location for the Model that has VS projects to import)
  • Layer (The layer to import the VS projects into)
  • LayerCode (The license code for development in the layer)
  • ModelName (The model to import the VS projects into)
  • BuildLogFolder (Specifies the log folder location.  Defaults to $env:TEMP.)
  • DetailedSummary (Parameter for msbuild.exe that creates a detailed summary in the log file.)
  • Verbosity (Parameter for msbuild.exe that allows you to set the level of logging)
  • Timeout (The value defaults to 30 minutes but can be overridden if desired)
  • SMTPServer (SMTP server to use to send the email)
  • MailMsg (Net.Mail.MailMessage object used to send the email)
  • VariablePath (Path to a file used to default the parameters/variables)

This function and the ImportVSProjects.proj can be found in Codeplex.  The steps of this function are:

  • Load the variables if a VariablePath parameter is used
  • Build the log files
  • Set up the arguments for the msbuild.exe call
  • Call msbuild.exe to import the projects
  • Send the success/fail message

I have also been cleaning up some of my PowerShell functions in preparation for putting them into a module.  I have made slight changes for uniformity and incorporated some suggestions from other AX users.  I have also incorporated my changes in the latest version of the DynamicsAXCommunity module (0.3.4).  I also have hopes that the next version will support my module with no customizations.  I will be updating past blogs posts in the near future to support my changes.

Combine-AXXPO custom PowerShell function

02 Thursday Jul 2015

Posted by in Powershell

≈ 4 Comments

Tags

, , , , , , , , ,

The Combine-AXXPO custom PowerShell function will create a single .xpo file from a directory containing .xpo files.  It is used in my build process to take the .xpos that are in TFS and combine them into a single file for import by model.

This is a fairly simple function that calls the Combine XPO tool.  As of AX 2012 R2, this tool is available in the Microsoft Dynamics AX\60\Management Utilities folder.  If you have been following my earlier posts, the path for the location is included in your profile already with .”C:\Program Files\Microsoft Dynamics AX\60\ManagementUtilities\Microsoft.Dynamics.ManagementUtilities.ps1″.  If you are using a version of AX earlier than 2012 R2, you will either have to add the tool here or include the location where it is in your profile.

This parameter takes between 2-11 parameters:

  • XpoDir (The directory holding the .xpos to be combined)
  • CombinedXpoFile (The output .xpo file)
  • SpecifiedXpoFile (A file that specifies which .xpos to combine)
  • NoDel (Allows DEL_ fields to be included for objects specified in the file)
  • utf8 (Sets the encoding of the output file to utf-8)
  • Threads (Specifies the number of threads to use for processing)
  • ExclusionsFile (A file that specifies which .xpos not to combine)
  • ViewsOnlyOnce (A parameter that can increase performance at the cost of reference errors)
  • SMTPServer (SMTP server to use to send the email)
  • MailMsg (Net.Mail.MailMessage object used to send the email)
  • VariablePath (Path to a file used to default the parameters/variables)

This function can be found in Codeplex.  The steps of this function are:

  • Load the variables if a VariablePath parameter is used
  • Generate the CombineXPOs.exe call based on the parameters passed
  • Run CombineXPOs.exe
  • Send the success/fail message

Restore-AXDatabase custom PowerShell function

18 Thursday Jun 2015

Posted by in Powershell

≈ 2 Comments

Tags

, , , , , , , , ,

I think I have officially become a bi-weekly blog.  My current schedule isn’t going to be able to support a weekly blog right now.  The Restore-AXDatabase custom PowerShell function will restore a SQL Server database from a backup file.  This is used in my custom build process to reset the build environment back to a “vanilla” instance.

A development build environment needs to be reset to an instance that is missing all of your build code before starting the new build.  It is possible to delete the build models and recompile/resync everything to get back to “vanilla” but it is significantly faster to make a database backup of the AX database and model database without your code present and then restore these.  It was taking me 5-20 minutes to delete models and 90-150 minutes to recompile and resync everything.  It generally takes me 2-5 minutes to restore the 2 databases.

The Restore-AXDatabase function takes between 2-6 parameters:

  • ServerInstance (Server and SQL instance where the DB is located.  Defaults to the local default instance.)
  • AXDBName (Name of the database to be restored.)
  • BackupFilePath (The backup file to be restored.)
  • AdditionalSQLRestore (This defaults to the values that I use for SQL 2012.  I have added it as a parameter so it could be changed if necessary.)
  • TimeOut (The value defaults to 10 minutes but can be overridden if desired)
  • SMTPServer (SMTP server to use to send the email)
  • MailMsg (Net.Mail.MailMessage object used to send the email)
  • VariablePath (Path to a file used to default the parameters/variables)

This function can be found in Codeplex.  The steps of this function are:

  • Load the variables if a VariablePath parameter is used
  • Set up the SQL command
  • Restore the DB using Invoke-SqlCmd
  • Send the success/fail message

There is some setup that needs to be done to use this function.  The first thing necessary it to load the PowerShell module that contains it.  I wrestled with adding this to the function itself but decided against it.  The reasoning is that if you are using an earlier version of SQL Server than 2012, you need to load the SQL PowerShell Snap-ins vs loading the module.

LoadSQLModule

I have added 3 lines to my profile to accomplish this.  The first and third line are added to make up for a side effect of loading the SQLPS module, it changes your location in PowerShell.  The Push-Location cmdlet pops your location onto a stack and the Pop-Location cmdlet retrieves it.  This allows you to load the SQLPS module without changing your location.

The other step that you will need to do is to get your build environment cleaned up and create backups of the AX database and the model database.  You are going to want to have only the models that aren’t part of your build installed in AX.  In my case, this would be my SYS, SYP, ISV & ISP layers.  I only have a single custom model in my VAR layer that I build.  I compile the AOT, IL and sync the database to make sure everything is ready.  I then take a backup of both my AX database and my model database.  Now I can restore both databases and use my Clean-AXArtifacts custom Powershell function to clean up all of the compiled IL and when I start my AOS back up, I have a working “vanilla” instance of AX for my build.

Sync-AXTFSWorkingFolder custom PowerShell function

04 Thursday Jun 2015

Posted by in Powershell

≈ 2 Comments

Tags

, , , , , , , ,

I apologize for missing a post last week.  The combination of a new home and a very busy work week, destroyed any possibility of getting a post out.  The Sync-AXTFSWorkingFolder function is one of 2 Team Foundation Server functions that I have for getting AX code.  This one is a lot less versatile than my other one but it is much more practical for a build process because it is light years faster.  We will explore my other function in a later post.

This function requires a little setup in your PowerShell profile.  The function needs to know the location of TF.exe and I handle this by adding it to the $env:path variable in my PowerShell profile.  There are many ways to solve this problem, this is just the one I’ve chosen.  TF.exe is available when you install Team Explorer for Visual Studio.  You should match your version of Team Explorer to the version of TFS that you are using.

The Sync-AXTFSWorkingFolder function takes between 1 and 6 parameters:

  • WorkingFolder (The working folder to sync from TFS)
  • Label (This value is used if you want to sync a label version instead of the current version)
  • SetLabel (Switch parameter that will allow you to label the current version if a unique Label parameter is passed with it)
  • LabelScope (Allows you to restrict the scope of the label in case there are multiple locations with the same label)
  • Comment (Allows you to set a comment on the label)
  • VariablePath (Path to a file used to default the parameters/variables)

This function can be found in Codeplex.  The steps of this function are:

  • Load the variables if a VariablePath parameter is used
  • Determine if a label was passed in
  • If so, set up the version and Label/LabelScope
  • Determine if the label exists in TFS
  • If so, sync the working folder with the label version
  • If not, get the current version and determine if the label needs to be set in TFS and set it
  • If no label is passed, sync the working folder with the current version
  • Send the success/fail message

This function assumes certain things.  It assumes that Visual Studio is installed with Team Explorer on the local computer.  It assumes that the user running the function has the working folder set up on the computer that the function is being run on.  It assumes that the user has permissions to read and set labels in TFS.  Lastly, it assumes that the path to TF.exe is known.  I also only run this function on my DevBuild server.  While this server has Team Explorer installed and configured with a working folder, my DevBuild AX environment is not connected to TFS.  Because of how AX natively works with TFS, I would not recommend running this function on a working folder that is used by AX as it could cause your AX instance to become out of sync with TFS.

Import-AXLabelFile custom PowerShell function

21 Thursday May 2015

Posted by in Powershell

≈ 2 Comments

Tags

, , , , , , , , ,

The Import-AXLabelFile PowerShell function allows you to import an AX label file into an environment.  While getting this function ready for the post, I realized that I had made a mistake in an earlier post.  My current build process was built to handle one layer and one model.  That was all I needed at the time and I had a limited amount of time to get it up and running.  It was always my plan to make this more generic however and to do that I needed to make my functions and build process able to work for multiple layers and models.  This required a few changes.

I needed a way to support what model is opened in AX for both this function and the Import-AXXPO function.  I have done that using the Model parameter.  Using the ConfigFile parameter to control layer and the Model file to control the model in that layer, individual calls have the ability to control the startup model versus using the “Startup model” system parameter that I had been using.  I have updated both the Start-AXAutoRun and Import-AXXPO functions to support this parameter as well as the blog posts regarding these functions.

The Start-AXAutoRun function takes between 5 and 8 parameters

  • ConfigPath (Client configuration for the chosen AX environment)
  • Model (The AX model to open in)
  • LabelFile (The label file to import)
  • Timeout (The value defaults to 10 minutes but can be overridden if desired)
  • SMTPServer (SMTP server to use to send the email)
  • MailMsg (Net.Mail.MailMessage object used to send the email)
  • AXVersion (The AX version.  It defaults to 6.)
  • VariablePath (Path to a file used to default the parameters/variables)

This function can be found in Codeplex.  The steps of this function are:

  • Load the variables if a VariablePath parameter is used
  • Validate the LabelFile parameter location
  • Start the AX client using the aldimport startup command
  • Wait for the timeout period for the AX client to exit
  • Send the success/fail message

This function is extremely similar to the Start-AXAutoRun function.  There are multiple commands to use to start AX and while AutoRun can do multiple things, it can’t import label files.  It is possible to reduce code duplication by dropping the Start-AXAutoRun function and replacing it with a very similar function that takes a command parameter for the AX -startupcmd parameter but I haven’t done that yet.  I hadn’t previously had a function for importing label files.  I was just doing this with a few lines of code in my build process.  Just one of the perils of building this weekly in my spare time.  Optimization through iteration.