Connecting to NAV Web Services from …

I promised to write some posts about how to connect to NAV Web Services from various other programming languages/platforms and I guess it is about time I kept my promise.

I will create a couple of posts on how to connect to NAV Web Services from:

  • PHP
  • Java
  • C# using Web Reference
  • C# using Service Reference
  • Javascript
  • Visual Basic using Web Reference
  • Visual Basic using Service Reference
  • Windows Mobile
  • Microsoft Dynamics NAV 2009SP1
  • and more…

Scenario

For all these I will create some code that:

  • Connects to NAV Web Services System Service and enumerates the companies in the database
  • Constructs a URL to the Customer Page (based on the first company in the list)
  • Get the name of Customer number 10000
  • Get the Customers in GB that have Location Code set to RED or BLUE.

Here is an example of the output of the PHP web site:

image

Note, that the code is not necessarily perfect PHP or Java code and the intention is not to teach people how to code in these languages (I would be a bad teacher) but more to overcome some of the obstacles when trying to work with NAV.

Authentication

The very first obstacle is authentication. As you probably know, NAV 2009 only supported SPNEGO (Negotiate) authentication and both PHP and Java doesn’t currently have any support natively for that.

In NAV 2009 SP1 we added a key in the Service Tier configuration file called WebServicesUseNTLMAuthentication. If you set this key to true the Web Services listener will only use NTLM authentication and as such be easier accessible from other systems.

The config file you need to modify is placed in the directory where the Service Tier executable is and is called CustomSettings.config. The section you are looking for is:

    <!–
Turns on or off NTLM authentication protocol for Web Services
false: Use SPNEGO (recommended)
true: Use NTLM only
–>
<add key=”WebServicesUseNTLMAuthentication” value=”true”></add>

Note that .net works with both values of this settings.

Enjoy

Freddy Kristiansen
PM Architect
Microsoft Dynamics NAV

Web Services changes in NAV 2009 SP1

NAV 2009 SP1 is being released later this year, so why write about it now?

The main reason is, that NAV 2009 SP1 comes out with a couple of changes, you might want to take into consideration when writing towards NAV 2009 Web Services.

Except for some performance enhancements, the major changes are:

New standard encoding of the Company name

I think one of the questions that has been asked the most is, how does my company name look in the URL: http://server:7047/DynamicsNAV/WS/<company>/

we all know that “CRONUS International Ltd.” becomes “CRONUS_International_Ltd”. Lesser known is it that “CRONUS USA, Inc.” becomes “CROUNS_USA_x002C__Inc” and there are a lot of special cases in this encoding algorithm.

In NAV 2009 SP1 we change to a standard Uri EscapeDataString function, meaning that

CRONUS International Ltd. becomes CRONUS%20International%20Ltd.

CRONUS USA, Inc. becomes CRONUS%20USA%2C%20Inc.

and

CRONUS+,æøå becomes CRONUS%2B%2C%C3%A6%C3%B8%C3%A5

fact is that you actually just can type in the Company name in the URL with the special characters in the browser and it will (in most cases) figure out to select the right company, even in Visual Studio when making your Web Reference (this won’t work if you have / or ? in the company name).

btw you can escape all the characters if you wish to

http://localhost:7047/DynamicsNAV/WS/%43RONUS%20International%20Ltd./Services

is a perfectly good company name – if you prefer Hex over characters.

This change has also affected the return values of the Companies function in SystemService – it now returns the un-encoded company names (= clear text). You can not any longer use the output from the companies function to build your URL – you need to escape the name.

Note: There is no backwards compatibility, trying to access webservices with a URL from NAV 2009 will fail, you need to change the company name encoding.

Schema changes in ReadMultiple and UpdateMultiple

Microsoft Infopath couldn’t modify multiple records using the NAV2009 page based Web Services due to a schema incompatibility. In NAV 2009 SP1 the schema changes for these methods. If you are using proxy based Web Service access (the add service or web reference in Visual Studio) you should just update the reference. If you are using XML Web Services you might have to modify the code used to parse the XML.

I will of course modify the samples on my blog where I use XPath to query the XML.

Updating records in Page based web services only updates the fields that you actually changed

The basics of XML Web Services is, that you send an XML document to a WebServices telling what you want to change. Visual Studio makes it easy to create a reference to a Web Service and get Strongly typed access to f.ex. Customers and Sales Orders through pages.

But how do we tell Web Services which fields actually changed?

For this, Visual Studio autogenerates a <field>Specified boolean property for all non-string fields from NAV and we will change ALL the fields, where <field>Specified is true or where a string is not NULL – NULL in a string value doesn’t mean clear the field, it means don’t update the field.

If you want to clear a field, set the value to String.Empty (“”).

In some cases this have caused problems. Primarily because when you read a customer record in order to change his name, it comes back from the Get function with all <field>Specified set to TRUE and all string fields have content. Changing the Name of a customer – writes the NAME and since the SEARCHNAME is included in the data sent to Web Services that gets updated as well (meaning that NAME and SEARCHNAME could be out of sync).

In NAV 2009 SP1 that has changed. Visual Studio of course still uses <field>Specified and string field <> NULL to determine what comes over the wire, but on the NAV side we only persist what you actually changed, so in NAV 2009 SP1 you can do:

Customer customer = custService.Read(“10000”);
customer.Name = “The Cannon Group, Inc.”;
custService.Update(ref customer);

and it will only update the name of the Customer. In NAV 2009 you would have to either set all the other fields in the customer to NULL or <field>Specified to false in order to get the same behavior – OR you could do like this:

Customer readCustomer = custService.Read(“10000”);
Customer updateCustomer = new Customer();
updateCustomer.Key = readCustomer.Key;
updateCustomer.Name = “The Cannon Group, Inc.”;
custService.Update(ref updateCustomer);

Which also will update only the name (just a small trick, instantiating a new Customer() will have all string fields set to NULL and <field>Specified for other fields set to false – and now we can just set the fields we want to change. Remember setting <field>Specified to true for all non-string fields.).

Note that this will of course work in SP1 as well and the advantage here is that you actually only send the new customer name over the wire to the Web Service.

Changes to how you Create and Updating Sales Orders through a page based Web Service

Actually the way you need to work with Sales Orders in NAV 2009 SP1 through a page based Web Service will also work in NAV 2009 – but the other way around is a problem. In NAV 2009 you could create a sales order with lines with just one call to Web Services, but in reality this didn’t work, you need to do this with a couple of roundtrips.

This is because application code (CurrPage.UPDATE) relies on one kind of transactional behavior (the new order is inserted and committed before any validation trigger starts), but Web Services enforce a different kind (every call to server is wrapped into one atomic transaction that is either committed or rolled back entirely – meaning that insert is not committed until all validation triggers passed).

I will create a special post on how to work with Sales Orders from Web Services – and try to show a way, which works for NAV 2009 SP1 (the same method will work for NAV 2009 as well – so you should consider this early).

Web Services doesn’t change the users default company

A Web Service consumer application would change the users default company in NAV 2009, but since Web Services doesn’t really use the notion of a default company for anything this seemed confusing – and made it impossible for a web service consumer application to call a web service method to request the users default company. In NAV 2009 SP1 – invoking a Web Service method does not change the default company for the user.

Blob fields in Page based Web Services are ignored

In NAV 2009 you couldn’t have a BLOB field on a page (image or whatever), which you exposed as a Web Service.

In NAV 2009 SP1, this has changed. This doesn’t mean that NAV transfers the content of the BLOB to the web service consumer – the field is just ignored.

If you want access to the value of the Blob you will need to write some code, which you can read something about here :

http://blogs.msdn.com/freddyk/archive/2008/11/04/transferring-data-to-from-com-automation-objects-and-webservices.aspx

Codeunits ignores methods with unsupported parameters

Like Pages with unsupported fields (BLOB), also codeunits can be exposed as Web Services even though they contain methods that use parameter types, that are not supported by Web Services. This could be streams, automation, blobs and more.

In SP1 you can connect to NAV Web Services from both PHP and Java

I won’t cover the details in this post, but it should be clear that NAV WebServices are accessible from PHP and Java come SP1. As soon as I have a build, which supports this – I will write a couple of posts on how to do this.

Enjoy

Freddy Kristiansen
PM Architect
Microsoft Dynamics NAV

Dynamic references to NAV Page Web Services in C#

Note: There is an updated post about Dynamic references to NAV Page Web Services here.

When creating the very first (never published) version of Edit In Excel, it was loosely coupled, meaning that I did not have any Web references in the project to the Customer Page, Vendor Page or other pages. I read the WSDL and used XPath to traverse the XML and build up structures and I was able to attach to any Page Web Service.

The code wasn’t nice, and I was afraid that I would confuse more people than necessary if I posted that source. So I decided to go with a version, where I had a Web References in VS for every page you can connect to Excel.

The caveat of this approach is that everytime you customize the Customer Page, you need to recompile your Edit In Excel solution – AND you need to do some work if you want to add additional pages. Wouldn’t it be nice if we could avoid this?

If you study the code in the Edit In Excel, you will find that the type-strong web references really aren’t used that much, the majority of the code uses the type of the service, the type of the field enumeration etc.

Last week I stumbled over a very interesting blog called crowsprogramming. A special thanks to the author for this post:

C# – Dynamically Invoke Web Service At Runtime

which in details shows how to ask the read the WSDL, build a service description and compile it into an assembly, and it basically consists of 3 methods:

/// <summary>
/// Builds an assembly from a web service description.
/// The assembly can be used to execute the web service methods.
/// </summary>
/// <param name=”webServiceUri”>Location of WSDL.</param>
/// <returns>A web service assembly.</returns>
public static Assembly BuildAssemblyFromWSDL(Uri webServiceUri)

/// <summary>
/// Builds the web service description importer, which allows us to generate a proxy class based on the
/// content of the WSDL described by the XmlTextReader.
/// </summary>
/// <param name=”xmlreader”>The WSDL content, described by XML.</param>
/// <returns>A ServiceDescriptionImporter that can be used to create a proxy class.</returns>
private static ServiceDescriptionImporter BuildServiceDescriptionImporter(XmlTextReader xmlreader)

/// <summary>
/// Compiles an assembly from the proxy class provided by the ServiceDescriptionImporter.
/// </summary>
/// <param name=”descriptionImporter”></param>
/// <returns>An assembly that can be used to execute the web service methods.</returns>
private static Assembly CompileAssembly(ServiceDescriptionImporter descriptionImporter)

When you call the first method with our NAV WebServices URL, it reads the WSDL, creates a CodeDom of the proxy classes, compiles the proxy and returns an assembly, which you can reflect over – all it takes is the following line of code:

// create an assembly from the web service description
Assembly webServiceAssembly = BuildAssemblyFromWSDL(
new Uri(“http://localhost:7047/DynamicsNAV/WS/CRONUS_International_Ltd/Page/Customer”));

Type weak

So, what can you really do with an Assembly in your hand…

You cannot use statements like:

if (customer.No == “10000”)

if the webservice isn’t added to the project. How should Visual Studio know that there is a No field in customer. So any Web References you add dynamically will only be there to reflect over and use via reflection, but you can do everything using reflection – it is just harder.

First of all – we can enumerate the public types in the assembly:

// Create Service Reference
Type[] types = webServiceAssembly.GetExportedTypes();
foreach (Type type in types)
Console.WriteLine(type.ToString());

running this, will output the following:

Customer_Service
Customer
Blocked
Copy_Sell_to_Addr_to_Qte_From
Application_Method
Reserve
Shipping_Advice
Customer_Filter
Customer_Fields

Which are the public types from an assembly, which is the proxy to a NAV Customer Page Webservice. Knowing that all pages follows the same pattern and that everything in Edit In Excel uses reflection over these classes anyway, it really became too compelling to rip out the Web Service References and make everything dynamic (that post will follow this one).

Working with reflection

I am not going to go into detail about how reflection works and what you can do with reflection, but I will show some examples of how to work with the dynamic assembly. First of all we want to create our service class:

Type serviceType = webServiceAssembly.GetType(“Customer_Service”);
object service = Activator.CreateInstance(serviceType);

if using static web references, this would be Customer_Service service = new Customer_Service();

Now we need to set the UseDefaultCredentials property to true:

PropertyInfo useDefaultCredentials = service.GetType().GetProperty(“UseDefaultCredentials”);
useDefaultCredentials.SetValue(service, (object)true, new object[] { });

in other words, get the info-class about the property based on the type, and call the setValue on the propertyinfo, specifying the object instance you want to set the value in, the value and an empty array, specifying that there are no parameters for this call.

Using static web references, this would be service.UseDefaultCredentials = true;

Next thing we want to do, is to call ReadMultiple and get all customers:

MethodInfo readMultiple = service.GetType().GetMethod(“ReadMultiple”);
object[] customers = (object[])readMultiple.Invoke(service, new object[] { null, null, 0 });

You see the picture – get the method info-class based on the type, and invoke the instance based method, specifying the instance and an object[] which contains the parameters you want to use.

In the static world this would be Customer[] customers = service.ReadMultiple(null, null, 0);

Now, we have an array of objects and the objects are of type Customer – but we don’t know about the customer type – only from reflection, so if we want to write the names of all customers we have to do something like:

Type customerType = webServiceAssembly.GetType(“Customer”);
PropertyInfo no = customerType.GetProperty(“No”);
PropertyInfo name = customerType.GetProperty(“Name”);
foreach (object customer in customers)
Console.WriteLine(no.GetValue(customer, new object[] { }) + ” ” + name.GetValue(customer, new object[] { }));

Which in a static implementation would be foreach(Customer customer in customers) Console.Writeline(customer.No + “ “ + customer.Name);

When to use dynamic Web References?

So, by now you got it – and yes, it is WAY easier to work with static Web References, inserted in the solution and using the strongly typed classes and methods, so when would you use dynamic Web References?

My answer to this is: Whenever you want to make something generic, where you can connect to different pages and/or where you don’t mind that the pages gets customized. In the scenario, where you have a fixed contract for requesting order information from a web service in NAV, there is absolutely no reason to use dynamic web references. In cases where you are connecting to a page based web reference where you have control over the page, it is easier (and probably safer due to type checking) to use the type strong web service access and maybe using LINQ with NAV Web Services from my last post.

But… – for something like Edit In Excel, dynamic Web References is a gift – and is really useful. I also think that it could be very useful with Bugsy’s Sharepoint sample – I need to investigate that…

Privileges

I was concerned whether stuff like this would require elevated privileges, but it turns out, that as long as the DLL you are creating / calling is going to run in the same context as your application, this doesn’t require anything. I tested this out running as non-administrator with UAC (Windows Vista User Access Control) turned on.

As usual, you can download the DynamicWebReference solution here.

The next thing I will do, is to extract some of the code from the Edit In Excel and create a set of classes, which makes working with dynamic web references easier. This will of course then be rolled back into the Edit In Excel R2.

Enjoy

Freddy Kristiansen
PM Architect
Microsoft Dynamics NAV

Using LINQ with NAV Web Services

In .NET 3.5 Microsoft released LINQ (Language INtegrated Query) – which is a way of writing select statements strongly typed inside your C# code.

Wouldn’t it be nice if we could leverage this technology when connecting to NAV Web Services?

I thought YES – so I set out to find out what it took…

The goal was to be able to replace code like this:

CustomerRef.Customer_Service cservice = new CustomerRef.Customer_Service();
cservice.UseDefaultCredentials = true;

CustomerRef.Customer_Filter filter = new CustomerRef.Customer_Filter();
filter.Field = CustomerRef.Customer_Fields.Location_Code;
filter.Criteria = “=YELLOW|=BLUE”;
CustomerRef.Customer_Filter[] filters = new CustomerRef.Customer_Filter[] { filter };

CustomerRef.Customer[] customers = cservice.ReadMultiple(filters, null, 0);

foreach (CustomerRef.Customer customer in customers)
{
// do stuff
}

with code like this:

CustomerRef.Customer_Service cservice = new CustomerRef.Customer_Service();
cservice.UseDefaultCredentials = true;

var cquery = from c in new FreddyK.NAVPageQuery<CustomerRef.Customer>(cservice)
where c.Location_Code == “YELLOW” || c.Location_Code == “BLUE”
select c;

foreach (CustomerRef.Customer customer in cquery)
{
// do stuff
}

Which I personally find more readable and has a lot of other advantages:

  1. Strongly typed criterias – if fields change type you get a compile error after updating the Web Reference
  2. Don’t have to think about CultureInfo (NumberFormat and DateFormat) when doing your criteria
  3. Don’t have to duplicate quotes for your search string
  4. Don’t have to convert booleans into Yes/No
  5. Separation of Query and Values in the Query
  6. and probably a lot more…

I always heard that you only needed 3 good reasons for doing something – so this should be more than enough.

Understanding LINQ

Lets look at the above code. The following

CustomerRef.Customer_Service cservice = new CustomerRef.Customer_Service();
cservice.UseDefaultCredentials = true;

initializes the Service (as usually) and

var cquery = from c in new FreddyK.NAVPageQuery<CustomerRef.Customer>(cservice)
where c.Location_Code == “YELLOW” || c.Location_Code == “BLUE”
select c;

creates a query (which contains an expression tree of the where clause) and

foreach (CustomerRef.Customer customer in cquery)
{
// do stuff
}

invokes the query (calls Web Services ReadMultiple) and returns the customers.

FreddyK.NAVPageQuery is a class, which is capable of acting as a LINQ provider for all pages exposed as Web Services.

The NAVPageQuery class is a generic class where you specify the type returned by the service – and you specify the service object as a parameter to the constructor. No need to say, that these two must match, else you will get an exception.

For a class to be usable in a LINQ query it needs to implement IQueryable<type> (which is a combination of IQueryable, IEnumerable<type> and IEnumerable) – and typically you will then have another class, which implements IQueryProvider. In my sample, I have implemented both interfaces in the NAVPageQuery.

Instead of listing the interfaces, I will list what happens when the query is build (when the line with var cquery = is executed):

  1. The NAVPageQuery gets instantiated
  2. LINQ calls the method IQueryProvider IQueryable.Provider to get the IQueryProvider class
  3. LINQ calls the method System.Linq.Expressions.Expression IQueryable.Expression to get the initial expression
  4. Now LINQ builds the expression tree and calls IQueryable<S> IQueryProvider.CreateQuery<S>(Expression expression) from which it expects a IQueryable<type> object.

Now we are done building the query and our variable cquery is set to be of type IQueryable<Customer>

When we then do a foreach on the cquery a little later, the following this happens:

  1. foreach calls into the IEnumerable interface IEnumerator<T> IEnumerable<T>.GetEnumerator() and ask for an enumerator
  2. In this method, I call the Execute method (TResult IQueryProvider.Execute<TResult>(Expression expression)) on the Provider (which is ‘this’)
  3. In the Execute method I run through the Expression tree and builds a number of Filter objects, collect them into an array of filters and call ReadMultiple.

You could argue that it would be a good idea to run through the expression tree while building the query – but the primary reason for NOT doing this is that you could imagine queries like:

decimal amount = 10000;

var cquery = from c in new FreddyK.NAVPageQuery<CustomerRef.Customer>(cservice)
where c.Balance_LCY > amount
select c;

In this case you want to separate building the query and creating the filters, since you could:

foreach (CustomerRef.Customer customer in cquery)
{
// do stuff
}

amount = 100000;
foreach (CustomerRef.Customer customer in cquery)
{
// do other stuff
}

this demonstrates the need for having the ability to decouple the query and the values used by the query and this is the reason for not creating the filter specs while building the query – but keeping the expression tree.

Another sample:

cquery = from c in new FreddyK.NAVPageQuery<CustomerRef.Customer>(cservice)
where c.Last_Date_Modified >= DateTime.Today.AddDays(-5)
select c;

At the time of execution, this query will give you customers modified the last 5 days (imagine building that one with filters easy)

Important things to notice

It is important to realize that NAVPageQuery doesn’t give you any functionality you didn’t have before. With the original Filter class you cannot OR things together (unless it is the same field) – you also cannot do that using LINQ (towards NAV Web Services)

If you try to type in a where clause like

where c.Location_Code == “YELLOW” || c.Country_Region_Code == “US”

It will fail – intentionally!

Of course I could have done client side filtering – but this would only fool the developer, since you would take the performance hit when calling ReadMultiple and you wouldn’t really notice – I wanted the developer to be aware when he makes clever or not so clever choices.

So – the where clause needs to be simple enough so that I can convert it into a filter specification.

The error will be thrown when you try to use the query and it will typically be an exception stating “Query too complex” (even though it might not be complex at all).

Another thing to notice is that AND’s and OR’s cannot be combined (as this isn’t possible in the filter spec):

where c.Location_Code == “YELLOW” && (c.Balance_LCY > amount || c.Location_Code == “BLUE”)

is not allowed, but

where c.Balance_LCY > amount && (c.Location_Code == “YELLOW” || c.Location_Code == “BLUE”) && c.No != “”

works just fine.

What about updating, deleting and creating?

Nothing new there – you still have your service object and the Customer objects that are returned from the LINQ query is the same object you would get from reading through Readmultiple directly.

This class is only designed to make queries towards web services easier and more error safe.

That’s all good – but how is it made?

For once, I am not going to go through how NAVPageQuery is created in detail – I have commented the source code pretty detailed and if you need to understand how it is build, you can look at the sourcecode combined with the information on how things are executed (from above) and it should be possible to decrypt the source:-)

If you download the solution .zip file here, you will find a class library with one class (NAVPageQuery) and a Console application called TestQuery. The console app. can be deleted if you don’t want it and it has a number of samples – some of which are listed above.

When downloading the test app. – you need to update the web references (so that they match your exposed Pages) – I know that I added the Last_Date_Modified to the Customer Page in order to have access to that field through web services – there might be more.

Questions and comments are welcome, I prefer to answer questions on mibuso instead of here – since it typically makes the answer available to more people.

Enjoy

Freddy Kristiansen
PM Architect
Microsoft Dynamics NAV