Web Services

Web services technology is a fast mode of distributing information between customers, suppliers, business partners and their various platforms. Web services are based on the "Service-Oriented Architecture" (SOA) model.

What is an API?

The term API stands for Application Programming Interface. The term can be used to describe the features of a library, or how to interact with it. Your favorite library may have "API Documentation" which documents which functions are available, how you call them, which arguments are required, etc.

However, these days, when people refer to an API they are most likely referring to an HTTP API, which can be a way of sharing application data over the internet. For example, Twitter has an API that allows you to request tweets in a format that makes it easy to import into your own application. This is the true power of HTTP APIs, being able to "mashup" data from multiple applications into your own hybrid application, or create an application which enhances the experience of using someone else's application.

A problem has started to arise when everyone starts implementing their own APIs. Without a standard way of naming URLs, you always have to refer to the documentation to understand how the API works. One API might have a URL like "/view_items" whereas another API might use "/items/all".

This is one of the reasons to develop Rest (or RESTful) Web services.

What is REST?

REST stands for Representational State Transfer. This is a term invented by Roy Fielding to describe a standard way of creating HTTP APIs. He noticed that the four common actions (view, create, edit, and delete) map directly to HTTP verbs that are already implemented: GET, POST, PUT et DELETE. (RFC2616 - Hypertext Transfer Protocol -- HTTP/1.1)

HTTP methods

There are technically 8 different HTTP methods. For DairyTrace, the following methods are implemented:
Method Description
GET
Request for the resource located at the specified URL
POST
Sends data (add) to the program located at the specified URL
PUT
Sends data (update) to the specified URL
DELETE
Deletes the resource located at the specified URL
HEAD
Request for the header of the resource located at the specified URL

Examples of REST

Let's look at a few examples of what makes an API "RESTful". Using our items example again...
  • If we wanted to view all items, the URL would look like this: GET http://example.com/items
  • Create a new item by posting the data: POST http://example.com/items
     Data:
      name = Name
  • To view a single item we "get" it by specifying that item's id: GET http://example.com/items/123
  • To validate the existence of a single item we send a HEAD command by specifying that item's id: HEAD http://example.com/items/123
  • Update that item by "putting" the new data: PUT http://example.com/items/123
     Data:
      name = New name
      color = black
  • Delete that item: DELETE http://example.com/items/123

Anatomy of a REST URL

You might have noticed from the previous examples that REST URLs use a consistent naming scheme. When you are interacting with an API, you are almost always manipulating some sort of object. In our examples, this is an Item. In REST terminology, this is called a Resource. The first part of the URL is always the plural form of the resource: /items

This is always used when referring to this collection of resources ("list all" and "add one" actions). When you are working with a specific resource, you add the ID to the URL: /items/123

This URL is used when you want to "view", "edit", or "delete" the particular resource.

HTTP Status Codes

Another important part of REST is responding with the correct status code for the type of request that was made. If you're new to HTTP status codes, here’s a quick summary. When you make an HTTP request, the server will respond with a code which corresponds to whether or not the request was successful and how the client should proceed.

There are five different levels of codes:
Code Description
1xx
Information
2xx
Success
3xx
Redirect
4xx
User Error
5xx
Server Error

Here is a list of the status codes used by REST APIs:
2xx – Success Codes Description
200
OK. Successful response. The actual response will depend on the request method used.
201
Created. The request has been fulfilled, resulting in the creation of a new resource.
204
No Content. The server successfully processed the request and is not returning any content..

4xx – User Error Codes Description
400
Bad Request. The server cannot or will not process the request due to an apparent client error (e.g., malformed request syntax, invalid request message framing, or deceptive request routing).
401
Unauthorized. The user does not have the necessary credentials.
404
Not Found. The requested resource could not be found but may be available in the future.
405
Method Not Allowed. A request method is not supported for the requested resource.
406
Not Acceptable. There is a validation error. Consult the response body to analyse the errors.
415
Unsupported Media Type. The request entity has a media type which the server or resource does not support.

HTTP message request field

Request Headers

The request header fields allow the REST service client to give additional information about the request. Here are the most commonly used request header by a client to a web service REST:
Request Header Fields Description
Acccept
Content-Types that are acceptable for the response.
Accept-Charset
Character sets that are acceptable
Accept-Language
List of acceptable human languages for response.
Authorization
Authentication credentials for HTTP authentication.
Content-Type
The MIME type of the body of the request (used with POST and PUT requests).
User-Agent
For reasons of safety, each custom application that will access the REST protocol must have an HTTP header type "User-Agent". This header will help us identify which applications are accessing our servers and it will be easier to offer support. We suggest including the name of your company, an application ID and version.

Example: User-Agent: Company-Apps-Version1

Connection with the Authorization header:

Example:
  • Connection information (before base64 encoding): Basic myUsername:myPassword
  • Connection information (after base64 encoding): Basic bXlVc2VybmFtZTpteVBhc3N3b3Jk
  • Final Authorization Header: Authorization: Basic bXlVc2VybmFtZTpteVBhc3N3b3Jk

Response Header

The response header fields allow the service to give additional information about the response. Here are the most commonly used response header used by a REST web service:
Response Headers Description
Allow
Fields to inform the client of the methods, character and / or languages valid associated with the requested resource.
Content-Length
Body length of the response.
Content-Type
Content type (format) of the body of the response.
Vary
Informs the customer on the criteria that were used to choose the representation of the requested resource.
Location
For 201 (Created) responses. The location is that of the new resource which was created by the application.
Warning
Carries additional information about the status or transformation of a message which might not be reflected in the message.
Status
This is the HTTP return code of the response (Status: HTTP/1.1 405 Method Not Allowed).
X-Service-Ati-Id
When a record is made. The unique identifier of the new record will be in this header (X-Service-Ati-Id: 12345).

API response formats

When you make an HTTP request, you can select the format that you want to receive. For example, making a request for a webpage, you want the format to be in HTML, or if you are downloading an image, the format returned should be an image. However, it's the server's responsibility to respond in the format that was requested. JSON has quickly become the format of choice for REST APIs. It has a lightweight, readable syntax that can be easily manipulated. So when a user of our API makes a request and specifies JSON as the format they would prefer:
GET /items Accept: application/json
Will return a JSON format :
[
    {
        id:123,
        name:'Simple Item'
    },
    {
        id:456,
        name:'My other item'
    }
]

Note that our API uses only JSON.

Synchronisation log

To get the new data, updated data and deleted data from DairyTrace's system you can use the "what's new" rest resource. The client just have to specify the start date and end date of the extraction. Note that if in the dowload you have no client_system_id you must add the link to our database throught a POST on the sync-options resource.

REST Call to get the log for the 2018 January month
GET /whats-new?account_id=12345&start_date=2018-01-01&end_date=2018-01-31

REST Call response example :
[
  {
    "action": "A",
    "log_date": "2018-01-18T14:15:53.946Z",
    "entity": "stakeholder",
    "dairy_trace_id": 54687958,
    "client_system_id": "client1_12345",
    "get_path": "/rest/stakeholders/54687958"
  },
  {
    "action": "A",
    "log_date": "2018-01-19T14:15:53.946Z",
    "entity": "stakeholder-role",
    "dairy_trace_id": 54687957,
    "client_system_id": "",
    "get_path": "/rest/stakeholders/54687958/roles/54687957"
  },
  {
    "action": "U",
    "log_date": "2018-01-19T14:15:53.946Z",
    "entity": "stakeholder-role",
    "dairy_trace_id": 54687959,
    "client_system_id": "client1_54321"",
    "get_path": "/rest/stakeholders/54687960/roles/54687959"
  }
]

In this case, in the month of January, there was a new stakeholder insertion and one role insertion for this stakeholder. For the first stakeholder-role insertion you must POST to sync-options :
POST /sync-options?account_id=12345
POST DATA : 
{
    "entity": "stakeholder-role",
    "client_system_id": "your_id_created",
    "dairy_trace_id": 54687957
}

Synchronisation options

You can also find DairyTrace's Unique identifier using the synchronisation option you provide for each record sent to DairyTrace.

REST Call to get the stakeholder id of a client id :
GET /sync-options?account_id=12345&entity=stakeholder&client_system_id=4444

REST Call response example :
[
  {
    "entity": "stakeholder",
    "client_system_id": 4444,
    "dairy_trace_id": 54687957
  }
]

In this case, the unique identifier for stakeholder_id in DairyTrace will be 54687957.

Searching in the REST API

It is possible to filter a list retrieved by a GET for any resources. Simply add a querystring to the REST call with the field name, the operator (URL Encoded) and the value you want to search for. To know if the field is searchable just consult the REST API documentation and search for field with (Searchable) beside the definition in the models.

Available search operator :

Operator (URL Encoded) Description Example
* (%2A) The asterisk is used to replace any sequence of characters. stakeholder_number=DAI* : This will retrieve the records with the stakeholder number starting with DAI.
| (%7C) The pipe is used between keyword to simulate an "or" statement. street_number=555|777 : This will retrieve the records with the street number 555 or 777.
< (%3C), > (%3E), <= (%3C%3D), >= (%3E%3D), The less than, greater than, less than or equal and greater than or equal will apply it's meaning to the records. street_number=%3E%3D555&street_number=%3C%3D777 : This will retrieve the records with the street number greater than or equal (>=) 555 and less than or equal (<=) 777.

Accounts

It is possible that your username is linked to multiple companies (stakeholder) that imports data in DairyTrace. If so you will have one of your account set as a main account. If you wishes to import data for another company, you must get the list of accounts and find the account_id of the company you want to import data for. Once this information is retrieved, you must pass it as the account_id querystring paramter of each futur REST calls. If the querystring parameter is not specified, the REST calls will be made as the main account.

REST Call to get accounts list :
GET /accounts

REST Call response example :
[
  {
    "stakeholder_id": 12345,
    "is_main": true,
    "stakeholder_number": "DAI1234567",
    "stakeholder_name": "DairyTrace Farm Inc.",
    "street_number": "555",
    "street_name": "Roland Therrien",
    "unit_number": "",
    "rural_route": "",
    "rr_group": "",
    "po_box": "",
    "city": "Longueuil",
    "postal_code": "J4H4E8",
    "province": "QC",
    "country": "CA",
    "segregation_province": "QC"
  },
  {
    "stakeholder_id": 54321,
    "is_main": false,
    "stakeholder_number": "DAI7654321",
    "stakeholder_name": "TraceDairy Farm Inc.",
    "street_number": "777",
    "street_name": "Roland Therrien",
    "unit_number": "",
    "rural_route": "",
    "rr_group": "",
    "po_box": "",
    "city": "Longueuil",
    "postal_code": "J4H4E8",
    "province": "QC",
    "country": "CA",
    "segregation_province": "QC"
  }
]

In this case, if you want to declare for TraceDairy Farm Inc. you will add ?account_id=54321 to your futur REST calls.

Stakeholders, Premises and Tags

Once you have chosen your account or using the main account you must navigate to an entity through the REST API Path. For example, to get the stakholders identifier to navigate in the stakeholders resources you must find the stakeholder_id.

REST Call to get stakeholders list :
GET /stakeholders?account_id=12345

REST Call response example :
[
  {
    "stakeholder_id": 12345,
    "stakeholder_number": "DAI1234567",
    "stakeholder_name": "DairyTrace Farm Inc.",
    "street_number": "555",
    "street_name": "Roland Therrien",
    "unit_number": "",
    "rural_route": "",
    "rr_group": "",
    "po_box": "",
    "city": "Longueuil",
    "postal_code": "J4H4E8",
    "province": "QC",
    "country": "CA",
    "segregation_province": "QC",
    "sync_options": {
      "client_system_id": "client1_12345"
    }
  },
  {
    "stakeholder_id": 54321,
    "stakeholder_number": "DAI7654321",
    "stakeholder_name": "TraceDairy Farm Inc.",
    "street_number": "777",
    "street_name": "Roland Therrien",
    "unit_number": "",
    "rural_route": "",
    "rr_group": "",
    "po_box": "",
    "city": "Longueuil",
    "postal_code": "J4H4E8",
    "province": "QC",
    "country": "CA",
    "segregation_province": "QC",
    "sync_options": {
      "client_system_id": "client1_54321"
    }
  }
]

In this case, if you want add a role for TraceDairy Farm Inc. you will add 54321 in the path to the POST on the right resource.
POST /stakeholders/54321/roles?account_id=12345
POST DATA : 
{
  "role_type": "Role_1",
  "premises_id": 1234567,
  "vehicle_id": 1234567,
  "start_date": "2017-01-01",
  "end_date": "2018-01-01",
  "sync_options": {
    "client_system_id": "client1_12345"
  }
}

If you wish to report promptly an event for a stakeholder, here are some basic steps to follow.

Here is a generic function to extract information about any resource. When you submit the web address of the resource in the $url parameter, you will get the HTTP response in the $result variable. This feature will eventually be used in further examples.

<?php
    function get($url)
    {
        $authUser = 'externe';
        $authPassword = 'Externe123';
    
        // cUrl Initialization
        $ch = curl_init();
        curl_setopt($ch, CURLOPT_URL, $url);
        
        // GET HTTP action
        curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
        curl_setopt($ch, CURLOPT_HEADER, 0);
        curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_NONE);
        
        // Add the User-Agent header to help identify your software
        curl_setopt($ch, CURLOPT_USERAGENT, 'Company_AppName_Version');
        
        // Authentication
        curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
        curl_setopt($ch, CURLOPT_USERPWD, $authUser.':'.$authPassword);
        
        // Execution
        $result['data'] = curl_exec($ch);
        $result['curl_error_code'] = curl_error($ch);
        $result['http_code'] = curl_getinfo($ch, CURLINFO_HTTP_CODE);
        curl_close($ch);
        unset($ch);
    
        return $result;
    }
?>
Here is a generic function to create information about any resource. When you submit the web address of the resource in the $url parameter, you will get the HTTP response in the $result variable. This feature will eventually be used in further examples.

<?php
    function post($url, $data)
    {
        $authUser = 'externe';
        $authPassword = 'Externe123';
        
        // cUrl Initialization
        $ch = curl_init();
        curl_setopt($ch, CURLOPT_URL, $url);
        
        // POST HTTP action
        curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'POST');
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
        curl_setopt($ch, CURLOPT_HEADER, 0);
        curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_NONE);
        
        // Add the User-Agent header to help identify your software
        curl_setopt($ch, CURLOPT_USERAGENT, 'Company_AppName_Version');
        curl_setopt($ch, CURLOPT_HTTPHEADER, 'Content-Type: application/json');
        
        // Authentication
        curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
        curl_setopt($ch, CURLOPT_USERPWD, $authUser.':'.$authPassword);
        
        // Execution
        $result['data'] = curl_exec($ch);
        $result['curl_error_code'] = curl_error($ch);
        $result['http_code'] = curl_getinfo($ch, CURLINFO_HTTP_CODE);
        curl_close($ch);
        unset($ch);
    
        return $result;
    }
?>
Here is a function allowing you to find the unique identifier of a stakeholder number using the generic function "get". Once the unique identifier is found, you can use it to navigate through all of the producer account resources.

<?php
    function getStakeholderId($stakeholderNumber)
    {
        $searchTerm = 'stakeholder_number=' . $stakeholderNumber;
        // You must url encode the search term
        $searchTerm = urlencode($searchTerm);
        
        $data = get('http://atq-mdelisleagtwebdev1/rest/stakeholders?search=' . $searchTerm . '&format=json');
        if($data['http_code'] == '404')
        {
            print('No record found');
            return false;
        }
        if($data['http_code'] != '200')
        {
            print('Error');
            return false;
        }
        $res = json_decode($data['data'], true);
        if(count($res['entries']) == 1)
        {
            return $res['entries'][0]['stakeholder_id'];
        }
        else 
        {
            print('More than one result');
            return false;
        }
    }
?>
Here is a function allowing you to submit declarations data for the stakeholder using the generic function "post".

<?php
    function postStakeholderDeclaration($stakeholder_id, $species, $declaration_data)
    {
        $data = post('http://atq-mdelisleagtwebdev1/rest/stakeholders/' . $stakeholder_id . '/declarations/' . $species . '?format=json', $declaration_data);
        if($data['http_code'] == '406')
        {
            print('Validation error, you must analyse the return values');
            return $data;
        }
        if($data['http_code'] != '201')
        {
            print('Other error');
            return $data;
        }
        // The event is successfully created
        return true;
    }
?>
Here is the call of the previous functions to submit an entry declaration for the PRO0000188 stakeholder. You must know the premises and identifier numbers.
<?php
    $stakeholder_id = getStakeholderId('PRO0000188');
    $declaration_data = array(
          'stakeholder_number' =>'PRO0000188',
          'declaration_type' =>'EntryMoveIn',
          'declaration_date' =>'2015-12-02 23:00:00',
          'identifier' =>'124000100001101',
          'premises_number' =>'QC1306072',
          'prov' =>'QC1439085'
      );
    $retour = postStakeholderDeclaration($stakeholder_id, 'cow', $declaration_data);
?>
[
    {
        id:123,
        name:'Simple Item'
    },
    {
        id:456,
        name:'My other item'
    }
]

If you wish to report promptly an event for a stakeholder, here are some basic steps to follow.

Imports System
Imports System.IO
Imports System.Net
Imports System.Text
Imports System.Net.Security
Imports System.Security.Cryptography.X509Certificates
Imports System.Xml

Module VBModule
 
    Sub Main()
        Dim Identification As String
        Dim username, password, myStakeholderId, successornot As String

        'Username and Password of Ontario LiveStock
        username = "livestock"
        password = "Simplitrace2016"
        'Convert to Base64
        Dim byt As Byte() = System.Text.Encoding.UTF8.GetBytes(username & ":" & password)
        Identification = System.Convert.ToBase64String(byt)
        'Find the stakeholder_id of Ontario LiveStock ENC number
        myStakeholderId = getStakeholderId("ENC0007458", Identification)

        'Convert the declaration to XML
        Dim strxml As StringWriter = new StringWriter()
        Dim writer As New XmlTextWriter(strxml)
        writer.Formatting = Formatting.Indented
        writer.WriteStartElement("root")
        writer.WriteElementString("stakeholder_number", "ENC0007458")
        writer.WriteElementString("declaration_type",  "EntryMoveIn")
        writer.WriteElementString("declaration_date",  "2017-01-01")
        writer.WriteElementString("premises_number",  "QC0000235")
        writer.WriteElementString("prov",  "QC0001258")
        writer.WriteElementString("identifier",  "124000100000007")
        writer.WriteElementString("weight",  "125")
        writer.WriteElementString("weight_unit",  "l")
        writer.WriteEndElement()
        
        'Send the declaration to ATQ for a cow
        successornot = postDeclaration(myStakeholderId, Identification, strxml, "cow")
    End Sub
    
    Function getStakeholderId(StakeholderNumber As String, Identification As String) As String
        Dim StatusCode, stakeholderInfo, search, url, stakeholder As String
        
        ' Create the URL for the request
        search = "search=stakeholder_number%3D" & StakeholderNumber
        url = "https://agtwebstaging.traceability.ca:443/rest/stakeholders?format=xml&language=en&" & search
        'Call ATQ Webservice to get the information about the stakeholder searched        
        httpGet(url, Identification, StatusCode, stakeholderInfo)
        'Analyse the XML response
        Dim strxml As StringReader = new StringReader(stakeholderInfo)
        Using reader As XmlReader = XmlReader.Create(strxml) 
          reader.MoveToContent()
          reader.ReadToDescendant("stakeholder_id")
          stakeholder = reader.ReadElementContentAsString()
        End Using
        'Return the stakehoder_id
        Return stakeholder
    End Function  
    
    Function postDeclaration(StakeholderId As String, Identification As String, PostData As StringWriter, Species As String) As String
        Dim StatusCode, returnBody, url, stakeholder As String
        'Create the url to create a declaration
        url = "https://agtwebstaging.traceability.ca:443/rest/stakeholders/" & StakeholderId & "/declarations/" & Species & "?format=xml"
        'Send to ATQ's WebService the DATA
        httpPost(url, Identification, StatusCode, returnBody, PostData)
        'Analyse the response, if the code is not 201, it failed and you have to analyse the XML
        If(StatusCode <> "201") Then
            Console.WriteLine(returnBody)
            Return "error"
        Else
            Return "success"
        End If
    End Function
    
    'This is a basic HTTP GET method, it can be used to call any resource of ATQ's webservice
    Sub httpGet(ByVal Url As String, ByVal Identification As String, ByRef StatusCode As String, ByRef stakeholderInfo As String)
        ServicePointManager.ServerCertificateValidationCallback = AddressOf AcceptAllCertifications
        ' Create a request for the URL. 
        Dim request As HttpWebRequest = WebRequest.CreateHttp(Url)
        request.UseDefaultCredentials = true;
        request.Headers.Add("Authorization", "Basic " & Identification)
        request.UserAgent = "Application V2.0" ' Here put your application information.
        ' Get the response.
        Dim response As WebResponse = request.GetResponse()
        ' Display the status.
        StatusCode = CType(response,HttpWebResponse).StatusCode
        ' Get the stream containing content returned by the server.
        Dim dataStream As Stream = response.GetResponseStream()
        ' Open the stream using a StreamReader for easy access.
        Dim reader As New StreamReader(dataStream)
        ' Read the content.
        ' Display the content.
        stakeholderInfo = reader.ReadToEnd()
        ' Clean up the streams and the response.
        reader.Close()
        response.Close()
        ServicePointManager.ServerCertificateValidationCallback = Nothing
    End Sub
    
    'This is a basic HTTP POST method, it can be used to call any resource of ATQ's webservice
    Sub httpPost(ByVal Url As String, ByVal Identification As String, ByRef StatusCode As String, ByRef returnBody As String, ByVal postData As StringWriter)
        ' Create a request for the URL. 
        ServicePointManager.ServerCertificateValidationCallback = AddressOf AcceptAllCertifications
        Dim response As WebResponse
        Dim dataStream As Stream
        Dim reader As StreamReader
        Try
            ' Create a request using a URL that can receive a post. 
            Dim request As HttpWebRequest = WebRequest.CreateHttp(Url)
            ' Set the Method property of the request to POST.
            request.Method = "POST"
            request.UseDefaultCredentials = true;
            request.Headers.Add("Authorization", "Basic " & Identification)
            request.UserAgent = "Application V2.0" ' Here put your application information.
            ' Create POST data and convert it to a byte array.
            Console.WriteLine(postData.ToString())
            Dim byteArray As Byte() = Encoding.UTF8.GetBytes(postData.ToString())
            request.ContentType = "application/xml"
            ' Set the ContentLength property of the WebRequest.
            request.ContentLength = byteArray.Length
            ' Get the request stream.
            dataStream = request.GetRequestStream()
            ' Write the data to the request stream.
            dataStream.Write(byteArray, 0, byteArray.Length)
            ' Close the Stream object.
            dataStream.Close()
            ' Get the response.
            response = request.GetResponse()
            ' Display the status.
            StatusCode = CType(response,HttpWebResponse).StatusCode
            ' Get the stream containing content returned by the server.
            dataStream = response.GetResponseStream()
            ' Open the stream using a StreamReader for easy access.
            reader = New StreamReader(dataStream)
            ' Read the content.
            returnBody = reader.ReadToEnd()
        Catch e As WebException
            If (e.Status = WebExceptionStatus.ProtocolError) Then
                response = e.Response
                Using (response)
                    Dim httpResponse As HttpWebResponse = CType(response, HttpWebResponse)
                    StatusCode = httpResponse.StatusCode
                    Try
                        reader = New StreamReader(response.GetResponseStream())
                        Using (reader)
                            returnBody = reader.ReadToEnd()
                        End Using
                    Catch ex As Exception
                        Console.WriteLine(ex)
                    End Try
                End Using
            End If
        Finally
            ' Clean up the streams.
            reader.Close()
            dataStream.Close()
            response.Close()
        End Try
        ServicePointManager.ServerCertificateValidationCallback = Nothing
    End Sub

    Public Function AcceptAllCertifications(sender As Object, certificate As X509Certificate, chain As X509Chain, sslPolicyErrors As SslPolicyErrors) As Boolean
        Return True
    End Function
    
End Module