Ada Based IoT Framework

Last updated on Sep 18, 2017
SUMMARY

The project integrates several new features to the AdaCore Drivers library to provide an IoT Framework based on existing LwIp implementation ported for the embedded STM32 Ethernet family of devices, by means of adapting and integrating existing network protocol Ada libraries adapted for the new LwIp port now it's possible to have HTTP server and MQTT client, a basic and classic hello world example code is provided for them as a starting point for complex and reliable embedded IoT and Ethernet Gateway solutions based on 100% Ada.

About the project.

When I first saw the contest I thought to do some IoT or even some advance application that relies on some networking protocol but soon I realize there was a lack for TCP stack for Ada Drivers Library, Ada-enet library provides UDP sockets but most of the well known protocols such HTTP needs TCP.  So this project ends up as a development project to support TCP and Application Layer protocol such HTTP.  Currently MQTT Client and HTTP Server offers basic functionality,  supporting code is provided as well as basic demonstrations snippets to support more complex applications.

Milestones.

I. Port the ipstack from AdaCore Spark2014 git to the embedded STM32 devices with Ethernet peripheral.

II. Adapt and port MQTT Client code from Dmitry A. Kazakov Simple Components Library.

III. Adapt and port the HTTP Server code from Simple Components, with underlying "socket" Server RAW lwip support for use in other Client/Server implementations.

IV. Provide example code of MQTT Client and Hello World HTTP Server for SMT32.

Detailed Description.

1. IP stack initialization. 

The Initialization consist of bringing the Ethernet driver up and lwIP stack. The first one is done similar to the ada-enet library, except that ipstack don't have (yet) DHCP protocol, so we fix the IP address static in code which is done in Initialize function of AIP.OSAL package, since ipstack also need to have IP addresses and in order to set the values in one place, the ipstack takes values from Ifnet record. The function If_Init is in charge of ipstack initialization and some discussion about it follows. 

   function If_Init return Err_T is
      Mac : LL_Address_Storage;
   begin
      Err := NOERR;
      Name := "st";
      NIF.Allocate_Netif (If_Id);
      if If_Id = NIF.IF_NOID then
         Err := ERR_MEM;
      end if;
      NIF.NIF_Set_Name (If_Id, Name);
      --  Mac (1) := Ifnet.Mac (1);
      for I in Integer range 1 .. 6 loop
         Mac (Net.Uint8 (I)) := Ifnet.Mac (I);
         Netif_MAC_Addr (Net.Uint8 (I)) := Ifnet.Mac (I);
      end loop;
      NIF.NIF_Set_Offload_Checksum (If_Id, NIF.IP_CS, True);
      NIF.NIF_Set_Offload_Checksum (If_Id, NIF.ICMP_CS, True);
      NIF.NIF_Set_Offload_Checksum (If_Id, NIF.UDP_CS, True);
      NIF.NIF_Set_Offload_Checksum (If_Id, NIF.TCP_CS, True);
      NIF.NIF_Set_LL_Address_Length (If_Id, LL_Address_Range'Last);
      NIF.NIF_Set_LL_Address (If_Id, Mac);
      NIF.NIF_Set_MTU (If_Id, Net.Uint16 (1500));
      IP := IPaddrs.IP4 (Ifnet.Ip (1), Ifnet.Ip (2), Ifnet.Ip (3), Ifnet.Ip (4));
      Mask := IPaddrs.IP4 (Ifnet.Netmask (1), Ifnet.Netmask (2), Ifnet.Netmask (3), Ifnet.Netmask (4));
      Broadcast := IPaddrs.IP4 (192, 168, 2, 255);
      Remote := IPaddrs.IP_ADDR_ANY;
      --  make static ARP for tests to isolate the issue with ARP append buffer
      declare
         remote_mac : aliased AIP.Ethernet_Address;
         remote_ip  : aliased AIP.IPaddrs.IPaddr;
         arp_err    : AIP.Err_T;
      begin
         remote_mac := (16#f8#, 16#32#, 16#e4#, 16#88#, 16#57#, 16#0c#);
         remote_ip := IPaddrs.IP4 (192, 168, 2, 5);
         ARP.ARP_Update
           (Nid             => If_Id,
            Eth_Address  => remote_mac,
            IP_Address      => remote_ip,
            Allocate        => True,
            Err             => arp_err);
         pragma Unreferenced (arp_err);
      end;
      --  define the callbacks
      NIF.NIF_Set_Input_CB (If_Id, AIP.IP.IP_Input'Access);
      NIF.NIF_Set_Output_CB (If_Id, AIP.ARP.ARP_Output'Access);
      NIF.NIF_Set_Link_Output_CB (If_Id, Net.MinIf.LL_Output'Access);
      NIF.If_Config (If_Id, IP, Mask, Broadcast, Remote, Err);
      return Err;
   end If_Init;

Notice that ipstack has prevision fo  the use of Checksum Offload Checks, which we take advantage of, so you have to set the flags accordingly using the NIF_Set_Offload_Checksum procedure. Also notice that we leave the callbacks as the original code, which was calling C code but now will call only Ada code instead.

You can notice that we were setting an initial value on the ARP list, this was done to do the tests without ARP packet going on and off there, so you can replace your host IP address there or just comment the code. I am not an expert on networking but it makes me some noise the fact that ipstack sends an ARP query when it receives an IP packet instead of trying to update the ARP cache using the data from the received packet, really don't know how  the standard should be, so suppose we have a server listening on port 7 (echo server), ipstack receive the SYN control packet and instead of replaying immediately with the SYN-ACK it put the SYN packet in the Unack Queue and sends an ARP Query for the Ip address received, then when ARP replay is received, it resumes the Three-Way handshake. 

During the debugging tests I solve a lot of things with original Three-Way handshake so I decide to isolate ARP using that snippet. 

So remember to change your desired IP address for STM32 there until DHCP is ready. (I will try to port DHCP from ada-enet one of these days :)

I started this project as I mentioned from ada-enet library so the "main" file remains called ping, which is something I will try to change later on,  and so the Receiver and Demos packages, so the call to Initialize is actually done in Demos package, i.e. AIP.OSAL.Initialize;

During my initial tests I notice some problem with my STM32F769I-DISCO board and the ada-enet library, I notice that some packets were lost during ping, it was concerning to me since I was not wanting to continue working with an underlying problem, the issue was reported on git of ada-enet without a final definitive solution, later on I realize the problem might be related to some conflict between Ethernet Stack and LCD, when I reported the issue I disable the D-cache of STM32 which seems to alleviate the problem, after some tests I decide to disable the LCD Refresh procedure that is called periodically  (which I used some way to do some debugging at the beginning), I mention this since you might notice some LCD function out there just in case you want to enable. 

2. Raw lwIP and it's callbacks.

Everything will be a callback since ipstack is based on RAW lwIP so you better get used to before driving nuts :)

To start receiving Ethernet Packets that will feed the ipstack a low level drivers was adapted. Original code uses Linux based Tap Interface, the embedded STM32 device Ethernet driver should be use to replace it. The great example code of Ada-enet ecosystem was used as a starting point, but reducing it to only the code required to gather the Ethernet frames since the ipstack has ARP, ICMP, UDP and TCP.  Basically the Net.Buffers package is used to gather Ethernet frames together with the Net.Interfaces package that servers to initialize the interface and provides the Send and Receive functions as well as other supporting  packages. 

Ada-enet uses a Receiver Task that loop forever delivering Ethernet frames as they arrive, that remains similar, so the call to Ifnet.Receive (Packet) will block until there is one frame available, after that we pass the buffer to a Low Level receive function that will copy  the data into ipstack buffers, so this version don't provide Zero Copy at the moment.

To maintain ipstack original structure as much as possible can be, the package AIP.OSAL.Single wraps different functions and procedures, Process_Interface_Events is in charge of allocating the buffer for the ipstack, the reason we don't call directly the low level interface is because the parent package, that is AIP.OSAL, holds the Interface id variable, originally ipstack allow an array of this interfaces Id but the embedded only has one interface and it doesn't make sense to have the array, so we declare If_Id in the definition file.

Returning to Process_Interface_Events function, it calls the  Low Level function LL_Input from Net.MinIf package under under src/dev folder to maintain a similar structure with the original code of Linux tap C driver. 

LL_Input it does the job of copying the frame into the necessary pbufs. Original code do some filtering of received frames, first attempt was to try activating the native filtering of STM32 Ethernet driver, since it can filter by Mac, but for some reason that didn't work as expected and as time goes by very fast we did the filtering by hand inside this function, only passing packets that match the Mac address or Broadcast packets (needed for ARP).

Here is the LL_Input function code.

   function LL_Input (Nid : AIP.NIF.Netif_Id;
                      Buf : in out Net.Buffers.Buffer_Type) return AIP.Buffers.Buffer_Id is
      p            : AIP.Buffers.Buffer_Id;
      q            : AIP.Buffers.Buffer_Id;
      size         : AIP.Buffers.Data_Length;
      Ether        : Net.Headers.Ether_Header_Access;
      DstMac       : Net.Ether_Addr;
      LL_Add       : AIP.LL_Address (1 .. 6);
      LL_Add_Len   : AIP.LL_Address_Range;
      Bail_Out     : Boolean := False;
      bufLen       : Standard.Integer;
      srcAdd       : System.Address;
      dstAdd       : System.Address;
   begin
--        pragma Unreferenced (Nid);
      Ether := Buf.Ethernet;
      if Net.Headers.To_Host (Ether.Ether_Type) = Net.Protos.ETHERTYPE_IP then
         --  If we got an IP packet verify the padding to alloc the correct size
         --  since Ada LwIp implementation uses Buffers.Tlen for some calcs
         declare
            IpHdr : Net.Headers.IP_Header_Access;
         begin
            IpHdr := Buf.IP;
            if Net.Headers.To_Host (IpHdr.Ip_Len) <= 46 then
               --  We set the size to alloc according to packet size
               size := Net.Headers.To_Host (IpHdr.Ip_Len) + 14;
            else
               size := Net.Buffers.Get_Data_Size (Buf, Net.Buffers.RAW_PACKET);
            end if;
         end;
      else
         size := Net.Buffers.Get_Data_Size (Buf, Net.Buffers.RAW_PACKET);
      end if;

      DstMac := Ether.Ether_Dhost;
      AIP.NIF.Get_LL_Address (Nid, LL_Add, LL_Add_Len);
      if (Uint_16 (size) < Uint_16 (LL_Add_Len)) then
         return AIP.Buffers.NOBUF;
      end if;
      --  Compare is packet is for us
      --  I know I can do this with Filter options of STM32 ...
      --  but I couldn't make it work, see line 370 of net-interfaces-stm32.adb
      for I in LL_Add'Range loop
         if LL_Add (I) /= DstMac (Standard.Integer (I)) then
            Bail_Out := True;
            exit;
         end if;
      end loop;
      --  Compare Broadcast if Packet Destination Addr match
      if Bail_Out then
         for I in 1 .. Standard.Integer (LL_Add_Len) loop
            if DstMac (I) /= Bcast (I) then
               return AIP.Buffers.NOBUF;
            end if;
         end loop;
      end if;
      AIP.Buffers.Buffer_Alloc (0, size, AIP.Buffers.LINK_BUF, p);
      if p /= AIP.Buffers.NOBUF then
         q := p;
         srcAdd := Net.Buffers.Get_Data_Address (Buf);
         loop
            bufLen := Standard.Integer (AIP.Buffers.Buffer_Len (q));
            dstAdd := AIP.Buffers.Buffer_Payload (q);
            AIP.Conversions.Memcpy (dstAdd, srcAdd, bufLen);
            q := AIP.Buffers.Buffer_Next (q);
            exit when q = AIP.Buffers.NOBUF;
            srcAdd := Net.Buffers.Get_Data_Address_Pos (Buf, Uint16 (bufLen));
         end loop;
      end if;
      Net.Buffers.Release (Buf);
      return p;
   end LL_Input;

Important to notice here  is the first part to determine the packet size that will be used to reserve the pbufs, it was noticed that the STM32 Ethernet driver has a check for minimum size of Ethernet frames, it does it for Tx and Rx as you will see in some tcpdump captures later on, for Tx is not concerning but for Rx it happens that it affects the ipstack if you reserve a size that includes the padding octects, which STM32 Ethernet drivers does for packets less than 60 bytes. I was trying to locate if Rx padding can be disable without success, so it seems that you can disable but it will affect Tx, so I never did the test and instead fix the size using the snippet before. The code uses some util function from ada-enet to extract the IP header length, which if less than 46 means padding will follow, this was only relevant for IP packets as ARP didn't bother about packet size, but TCP ipstack code will fail if wrong.

Receiver Task.

The receiver endless task will dispatch packets depending on it's type, for now we have to discern from ARP and IP packets, here is the Receiver task.

task body Controller is
   use type Net.Uint64;
   use type Net.Ether_Addr;

   Buf     : AIP.Buffers.Buffer_Id;
   Ethhr   : System.Address;
   Ftype   : Net.Uint16;
   Packet  : Net.Buffers.Buffer_Type;
   Err     : AIP.Err_T;
begin
   --  Wait until the Ethernet driver is ready.
   Ada.Synchronous_Task_Control.Suspend_Until_True (Ready);
   AIP.IO.Put_Line ("Receiver Task Starts");
   --  Loop receiving packets and dispatching them.
   loop
      if Packet.Is_Null then
         Net.Buffers.Allocate (Packet);
         exit when Packet.Is_Null;
      end if;
      if not Packet.Is_Null then
         --  Block until there is a packet to process
         AIP.OSAL.Ifnet.Receive (Packet);
         Buf := AIP.OSAL.Single.Process_Interface_Events (Packet);
         if Buf /= AIP.Buffers.NOBUF then
            Ethhr := AIP.Buffers.Buffer_Payload (Buf);
            Ftype := AIP.EtherH.EtherH_Frame_Type (Ethhr);
            case Ftype is
               when AIP.EtherH.Ether_Type_IP =>
                  AIP.Buffers.Buffer_Header (Buf, AIP.S16_T (-14), Err);
                  if AIP.Any (Err) then
                     AIP.Buffers.Buffer_Blind_Free (Buf);
                  else
                     AIP.OSAL.Single.Process_Input (Buf);
                  end if;
               when AIP.EtherH.Ether_Type_ARP =>
                  AIP.OSAL.Single.Process_Arp (Buf);
               when others =>
                  AIP.Buffers.Buffer_Blind_Free (Buf);
            end case;
         end if;
      end if;
   end loop;
   AIP.IO.Put_Line ("Problem with Net Stack Packet allocation");
end Controller;

One thing to notice here is that when an error is encountered it's necessary to Free the LwIP buffer. For other cases the Free will be done inside the calling procedures. Before moving to discuss how ipstack handle the packets, let's finish the low level output procedure.

The same package Net.MinIf that has the Low Level Input seen before has the Low Level Output procedure, and actually does the inverse, takes ipstack pbufs and copy it's data to a Net.Buffers for Ethernet Tx. The difficult part was to realize how to get track of next position to write in the Net.Buffers Packet which is done using the function Get_Data_Address_Pos, after all data have been copied, it only remains to set the Length and call the Send procedure of Net.Interfaces.STM32 package to deliver it.

The low level output procedure will be the called by ipstack using the AIP.NIF package Link_Output procedure which is done only in AIP.ARP package, the other Output procedure that will send IP packets is actually ARP_Ouput, because the Ethernet information is filled by info in ARP table.

IP Packets.

The receiver dispatch IP packets by calling AIP.OSAL.Single.Process_Input (Buf) procedure, which indeed calls the configured callback IP_Input procedure in AIP.IP package. 

IP_Input do some sanity checks on the IP Header itself, among them it checks the IP address is the configured, original ipstack has some (To Be Defined or TBD) functions to forward which does not make sense here with only one interface so it's always disable (Enable_Forwarding flag is False).

At the end it dispatch the packet by calling Dispatch_Upper, this procedure call the appropriately procedure for UDP, TCP or ICMP. The first thing you want to try is ICMP as you guess. AT this point if IP address is configured correctly it should work out of the Box.

TCP Packets.

TCP packets are handle by TCP_Input procedure. It performs some sanity checks on TCP Header as well, but if packet is correct it try to finds the PCB to deliver it, this is done by PCBs.Find_PCB in AIP.PCBs package. PCB are named the Protocol Control Blocks and they are share between TCP and UDP, for sure you know better than me about this, but in short you will have one PCB for each "connection" you have, let's say you want to connect to a server, ipstack will reserve an available PCB from a PCB array for that connection during it's lifetime, so it uniquely identify it, perhaps that was an oversimplification in LwIP terms but someone that ones to works at upper layer don't have to know every detail of it.

If there is a PCB for the received TCP packet, the TCP_Input calls TCP_Receive to further process it, otherwise it might send a RST control segment to peer to inform about it.  

TCP_Receive procedure handles the TCP packet by delivering it to the application when needed and changing TPCB State.

It might be boring at some point this discussion but even worse was to make it work (actually frustrating is more appropriately) since original ipstack seems to have some blackholes, for example after the ICMP test works I tried the Echo Server that was already in ipstack code, which "works" when tested with a simple python client that connects to port 7, send "Hello World" and disconnect, the console shows the received Echo message but tcpdump capture was concerning. See known issue for more details.

Here is a screenshot of such a first capture.

Echo Three-Way wrong

As you can see the Server was not handling correctly the Three-Way handshaking, actually it was even worse since after the client disconnect by sending the FIN-ACK server sends an empty payload packet, in short it was needing some inspection.

We are not going to discuss all the lines of code of this function at this moment since it's very long, rather we will progress with the project documentation and come back to highlight certain changes and explanation that is done in this and other aip.tcp procedures and functions. 

Important right now before moving on is to hightlight the way the TCP stack inform the App layer about reception of packets. It does by mean of TCP_Event procedure, which actually insert the Event and PCB from the stack and call (if defined by the Application layer) the assigned callback procedure, let's see how it works in the case of the simple echo server.

Echo Server.

Echo server is really simple, all the code is located in a single package under src/services called raw_tcp_echo.adb. The server initialization is done in demos.adb by calling RAW_TCP_Echo.Init.

The Init function is shown below

   procedure Init with
     Refined_Global => (In_Out => ESP)
   is
      Pcb : AIP.PCBs.PCB_Id;
      Err : AIP.Err_T;

   begin
      --  Initialize the application state pool, then setup to
      --  accept TCP connections on the well known echo port 7.

      Init_ES_Pool;

      AIP.TCP.TCP_New (Pcb);
      if Pcb = AIP.PCBs.NOPCB then
         Err := AIP.ERR_MEM;
      else
         AIP.TCP.TCP_Bind
           (PCB        => Pcb,
            Local_IP   => AIP.IPaddrs.IP_ADDR_ANY,
            Local_Port => 7,
            Err        => Err);
      end if;

      if Err = AIP.NOERR then
         AIP.TCP.TCP_Listen (Pcb, Err);
         pragma Assert (AIP.No (Err));
         AIP.TCP.On_TCP_Accept
           (Pcb, RAW_TCP_Callbacks.To_CBID (ECHO_Process_Accept'Access));
      end if;

   end Init;

The function TCP_New (Pcb) creates the server Pcb, after that a bind and listen on that Pcb follows. The Accept callback is set for the Pcb using On_TCP_Accept procedure. 

More interesting is what is done in the accept callback shown below.

   -------------------------
   -- ECHO_Process_Accept --
   -------------------------

   procedure ECHO_Process_Accept
     (Ev  : AIP.TCP.TCP_Event_T;
      Pcb : AIP.PCBs.PCB_Id;
      Err : out AIP.Err_T)
   is
      pragma Unreferenced (Ev);

      Sid : ES_Id;

   begin
      ES_Alloc (Sid);

      if Sid = NOES then
         Err := AIP.ERR_MEM;
         Raise_Exception
           (Data_Error'Identity,
            "Alloc Sid Failed!");
      else
         ESP (Sid).Kind := ES_ACCEPTED;
         ESP (Sid).Pcb  := Pcb;
         ESP (Sid).Buf  := AIP.Buffers.NOBUF;

--           AIP.IO.Put_Line ("New PCB Accept: Sid");
--           AIP.IO.Put_Line (Sid'Image);

         AIP.TCP.TCP_Set_Udata (Pcb, ESP (Sid)'Address);

         AIP.TCP.On_TCP_Sent
           (Pcb, RAW_TCP_Callbacks.To_CBID (ECHO_Process_Sent'Access));
         AIP.TCP.On_TCP_Recv
           (Pcb, RAW_TCP_Callbacks.To_CBID (ECHO_Process_Recv'Access));
         AIP.TCP.On_TCP_Abort
           (Pcb, RAW_TCP_Callbacks.To_CBID (ECHO_Process_Abort'Access));
         AIP.TCP.On_TCP_Poll
           (Pcb, RAW_TCP_Callbacks.To_CBID (ECHO_Process_Poll'Access), 500);

         AIP.TCP.TCP_Accepted (Pcb);

         Err := AIP.NOERR;
      end if;
   end ECHO_Process_Accept;

The creators of the ipstack were cleaver to create a record to hold the state of each connection, it's actually an array since server can have more than one client connected at a given time. The array type is Echo_State and consists of 

   type State_Kind is
     (ES_FREE, ES_READY, ES_ACCEPTED, ES_RECEIVED, ES_CLOSING);
   type Echo_State is record
      Kind : State_Kind;
      Pcb  : AIP.PCBs.PCB_Id;
      Buf  : AIP.Buffers.Buffer_Id;
      Err  : AIP.Err_T;
   end record;

The beauty of the stack is the way to recover the state when a callback is done, the state array is actually an array that lives inside this file, for the echo tcp demo it doesn't seems like a big deal but later on you will see how we take advantage of it. The ipstack IPCBs ins an array of IP_PCB, a record that holds information such remote/local ip and port, connection status and link type among others, but there is an item of the record called Udata which is actually a pointer to the application data, you guess, in out TCP echo that data is  our Echo_State record. The way is assigned  by the application is using the function TCP_Set_Udata shown in the accept callback after the Pcb has been reserved in our Echo_State array ESP, the demo allows to have 4 records on this array but it's a matter of resources to have more.

If the server accepts the connection, it set up the callbacks for Sent, Receive, Abort and Poll. The last one is important to notice, it's a callback that is executed periodically by TCP so you might ask, where is the timer?

TCP Timers.

TCP has two functions called periodically by means of a Timer, the functions are configured in TCP_Init shown below.

   --------------
   -- TCP_Init --
   --------------

   procedure TCP_Init with
     Refined_Global => (Output => (All_PCBs,
                                   Boot_Time,
                                   IPCBs,
                                   TCP_Ticks,
                                   TPCBs))
   is
   begin
      --  Record boot time to serve as local secret for generation of ISN

      Boot_Time := Time_Types.Now;
      TCP_Ticks := 0;

      --  Initialize all the PCBs, marking them unused, and initialize the list
      --  of bound PCBs as empty.

      IPCBs    := TCP_IPCB_Array'(others => PCBs.IP_PCB_Initializer);
      TPCBs    := TCP_TPCB_Array'(others => TCP_PCB_Initializer);
      All_PCBs := TCP_PCB_Heads'(others => PCBs.NOPCB);

      --  Allocate and Set frequency of TCP timers

      Timers.TID_Alloc (Timers.TIMER_EVT_TCPFASTTMR, TCP_Fast_Timer'Access);
      Timers.TID_Alloc (Timers.TIMER_EVT_TCPSLOWTMR, TCP_Slow_Timer'Access);
      Timers.Set_Interval (Timers.TIMER_EVT_TCPFASTTMR, TCP_Fast_Interval);
      Timers.Set_Interval (Timers.TIMER_EVT_TCPSLOWTMR, TCP_Slow_Interval);
   end TCP_Init;

I modify the original AIP.Timers since it was kind of limited in the sense that it was tedious and time consuming and more error prone to add a Timer that was not in a Fixed list of Timers, so I spend some time to make it flexible and so any application service that requieres a Timer might used it, it was actually that the reason I did it, since MQTT protocol needs a Timer (more on that later on). The function TID_Alloc allocates a new Timer by associating it with a callback. You might notice I still use the List of Fixed Timers, but indeed there is another function that uses Index values and so you reserve it without knowing which index of the Timer it will assigns to.

The Timer is not started until you assign an interval by means of Set_Interval. 

One important thing is that the Timers runs in a separate task called Periodic of Os_Service package. This file is locate inside the demos/utils board since I didn't find a better place for it.  Periodic dedicate itself to measure time and call Process_Timers every TIMER_PERIOD as shown below. 

   task body Periodic is

      Prev_Clock, Clock  : AIP.Time_Types.Time := AIP.Time_Types.Time'First;
      timer_deadline : Ada.Real_Time.Time;

      Poll_Freq : constant := 50;

   begin
      pragma Unreferenced (Prev_Clock, Poll_Freq);
      Ada.Synchronous_Task_Control.Suspend_Until_True (Ready);
      Clock := AIP.Time_Types.Now;
      timer_deadline := Ada.Real_Time.Clock;
      loop
         delay until timer_deadline;
         Clock := AIP.Time_Types.Now;
         AIP.OSAL.Single.Process_Timers (Clock);
         timer_deadline := timer_deadline + TIMER_PERIOD;
      end loop;
   end Periodic;

The Timers works very good but callbacks should not take too long  to return or it will start to fail, in the future a better approach of having the callbacks fired should be investigated. 

Echo Server callbacks. 

The callbacks has no mean to know the actual state of the given connection unless once inside it  by some way it finds which Echo State Pool corresponds the call, so this was the reason we assign Application data to the IPCB as shown before, so to retrieve it the callback does 

Es : Echo_State;
for Es'Address use AIP.TCP.TCP_Udata (Pcb);

This way the execution of code is ruled by the current state i.e., the more obvious usage is during reception, for example if reception callback is called without any buffer in the Event parameter (TCP_Event_T) and there are no more buffers to process, we might call Close on the Pcb but if there is pending data to process, the server sends it and delegates the Close to the Polling callback since it already set the state as ES_CLOSING. The code is shown below for clarity.

   -----------------------
   -- ECHO_Process_Recv --
   -----------------------

   procedure ECHO_Process_Recv
     (Ev  : AIP.TCP.TCP_Event_T;
      Pcb : AIP.PCBs.PCB_Id;
      Err : out AIP.Err_T)
   is
      Es : Echo_State;
      for Es'Address use AIP.TCP.TCP_Udata (Pcb);

   begin
      if Ev.Buf = AIP.Buffers.NOBUF then

         --  Remote host closed connection. Process what is left to be
         --  sent or close on our side.

         Es.Kind := ES_CLOSING;

         if Es.Buf /= AIP.Buffers.NOBUF then
            Echo_Send (Pcb, Es);
         else
            Echo_Close (Pcb, Es);
         end if;

      else
         --  Signal TCP layer that we can accept more data

         case Es.Kind is
            when ES_ACCEPTED =>

               Es.Kind := ES_RECEIVED;
               Es.Buf  := Ev.Buf;
               AIP.Buffers.Buffer_Ref (Ev.Buf);
               Echo_Send (Pcb, Es);

            when ES_RECEIVED =>

               --  Read some more data

               if Es.Buf = AIP.Buffers.NOBUF then
                  AIP.Buffers.Buffer_Ref (Ev.Buf);
                  Es.Buf := Ev.Buf;
                  Echo_Send (Pcb, Es);

               else
                  AIP.Buffers.Buffer_Chain (Es.Buf, Ev.Buf);
               end if;

            when others =>

               --  Remote side closing twice (ES_CLOSING), or inconsistent
               --  state. Trash.

               AIP.TCP.TCP_Recved (Pcb, AIP.Buffers.Buffer_Tlen (Ev.Buf));
               Es.Buf := AIP.Buffers.NOBUF;

         end case;

      end if;

      Err := AIP.NOERR;
   end ECHO_Process_Recv;

As you can see, Echo server is simple, it just call Echo_Send on data received by actually reusing the Pcb since data it's not even changed. That will not be the case of other servers you and I might have in mind.

One point here that I will take the opportunity to highlight.  If you see the tcpdump capture of the TAP demo of ipstack that works, you will notice something odd. 

ipstack TCP Echo tcpdump capture

The Server don't replay the received Data with an Ack, as far as I know that is not correct but it might do the Job. After I was having issues with HTTP server I decide to get's my hands on this issue too. 

The original stack calls the procedure AIP.TCP.TCP_Recved (Pcb, Plen); to adjust the receive window once the application has consumed the data but leverages the Ack send to the stack state machine, amd for some reason it was missing. 

Among several modifications I did, I added sort of Ack Now inside the function taking care to remove the Buffer from the Unack_Queue after that. Actually when looking at this I spend some time reading the C implementation of lwip mqtt where there is such Ack Now function,  I personally think there might be a better or different way to correct the problem but this one works so far as you can see in the following capture of the embedded STM32 ipstack Echo server.

STM32 TCP Echo capture

Here is the TCP_Recved snippet

   ----------------
   -- TCP_Recved --
   ----------------

   procedure TCP_Recved
     (PCB : PCBs.PCB_Id;
      Len : AIP.U16_T)
   with
     Refined_Global => (In_Out => TPCBs)
   is
      Seg : Buffers.Buffer_Id;
      Thdr : System.Address;
   begin
      --  Open receive window now that application has consumed the data

      TPCBs (PCB).RCV_WND := TPCBs (PCB).RCV_WND + AIP.M32_T (Len);

      if Len > 0 then

         Buffers.Buffer_Alloc
           (Size   => TCPH.TCP_Header_Size / 8,
            Offset => Inet.HLEN_To (Inet.IP_LAYER),
            Kind   => Buffers.SPLIT_BUF,
            Buf    => Seg);

         Thdr := Buffers.Buffer_Payload (Seg);

         TCPH.Set_TCPH_Data_Offset (Thdr, 5);
         --  No options present

         TCPH.Set_TCPH_Seq_Num  (Thdr, TPCBs (PCB).SND_NXT);

         TCPH.Set_TCPH_Urg (Thdr, 0);
         TCPH.Set_TCPH_Psh (Thdr, 0);
         TCPH.Set_TCPH_Syn (Thdr, 0);
         TCPH.Set_TCPH_Fin (Thdr, 0);
         TCPH.Set_TCPH_Rst (Thdr, 0);  --  Force Reset since we are embedded
         TCPH.Set_TCPH_Ack (Thdr, 1);

         Buffers.Set_Packet_Info (Seg, Thdr);
         TCP_Send_Segment
           (Tbuf       => Seg,
            IPCB       => IPCBs (PCB),
            TPCB       => TPCBs (PCB));

         Buffers.Remove_Packet
           (Buffers.Transport, TPCBs (PCB).Unack_Queue, Seg);

         Buffers.Buffer_Blind_Free (Seg);

      end if;
   end TCP_Recved;

The procedure TCP_Output can also be used to Ack Now but if it also Flush the Send_Queue, anyway this might be a discussion on github with other folks that know this better than me.

When the echo server App close the connection it also release it's Echo State Pool (ESP) by calling ES_Release procedure, which basically marks the connection as ES_FREE state and null address the Application Data.

So now the Echo_Send first call TCP_Recved and after that it writes directly using TCP_Write.

Three-Way Handshaking.

Soon as I started working on the mqtt client, I realize the tcp stack was failing to act as a client, but I will discuss those modifications in a moment. Original code of server seems to implement the active close but not the passive close or at least I did't find a way to make it work out of the box, so start looking at what a passive close should look like from server side. It happens that once the FIN is received the server should go to Close_Wait State and send it's corresponding FIN, after that it transition to Last_Ack.

Here is a very helpful diagram of how this.

TCPIP_State_Transition_Diagram.pdf

I added the Last_Ack  State check to the TCP_Receive since when this Ack is received it only concerns the TCP stack actions, and so it don't need to be inform application about it. The action consist of transition to Close state and release/reset the Pcb for later reuse, the snippet is

when Last_Ack =>
 --  Passive Three Way Last Ack processing
 Set_State (PCB, Closed);
 TCP_Free (PCB);
 Err := AIP.NOERR;

It works because I also did a modification in TCP_Close, I will bring it here the few snippets out there that plays on this.

First all packets with data are passed to the application using this on aip.tcp.adb

 if New_Data_Len > 0 then
    Deliver_Segment (PCB, Seg.Buf);
 end if;

but for some reason I can really understand (that makes me want to rewrite the code) the New_Data_Len variable counts theFIN bit, that is, the FIN if received add one "1" to this variable so it enter there for processing, but in fact nothing is done unless the packet has some data, since inside Deliver_Segment procedure the application callback is done if the segment len is not zero, and then nothing else happens. SO is in fact the next piece of code of TCP_Receive that informs our application of the FIN received, that is (the relevant part right now)

if TCPH.TCPH_Fin (Seg.Thdr) = 1 then
  case TPCBs (PCB).State is
     when Closed | Listen | Syn_Sent =>
        null;

     when Established | Syn_Received =>
        --  Notify connection closed: deliver 0 bytes of data.
        --  First transition to Close_Wait, as the application
        --  may decide to call Close from within the TCP_Event
        --  callback.

        Set_State (PCB, Close_Wait);
        TCP_Event
          (Ev   => TCP_Event_T'(Kind => TCP_EVENT_RECV,
                                Len  => 0,
                                Buf  => Buffers.NOBUF,
                                Addr => IPaddrs.IP_ADDR_ANY,
                                Port => PCBs.NOPORT,
                                Err  => AIP.NOERR),
           PCB  => PCB,
           Cbid => TPCBs (PCB).Callbacks (TCP_EVENT_RECV),
           Err  => Err);

The first thing is to change it's state as the diagram told us to do, next call application layer. And now the TCP_Close that follows (relevant part only shown)

when Close_Wait =>

	--  Transition to LAST_ACK after sending FIN

	TCP_Fin (PCB, Err);
	if AIP.No (Err) then
	   Set_State (PCB, Last_Ack);
	   --  Purge any remaining Buffer from Unack Queue
	   TCP_Purge (PCB);
	   Flush := False;
	end if;

The modification I include is a function that Purge PCB queues here, and thou just waiting the Last_Ack to Free the PCB as seen before. The reason of this TCP_Purge is because original stack calls TCP_Fin which sends the FIN but also includes the the packet in the Unack_Queue.  This was giving me problems of retransmissions, but it has the possibility that if Ack is not received it will not send again the FIN, so really this need further revisions. One problem I faced is debugging this things, sometimes you need a real fast printf that don't affect execution time and semihosting on ST-Link is all but that.

3. Working on top of ipstack (raw lwip).

Make it smart. 

The echo server by itself it's not fun nor very useful alone but it motivate me to continue working on this. Since contest time frame was really short (specially for me as I started in mid July) and the fact that I am really new to Ada world, I was not willing to reinvent the wheel, so I start looking at existing code for Application Layer protocols, specially I was looking for HTTP which is nice and really necessary to have. I found a few stacks out there but none made for the bare metal embedded world or even that I was able to compile using the Ada ARM version so I spend some time figuring out which one would be feasible to port. Then I found the work of Dmitry A. Kazakov, which really took my attention because it provides not only HTTP but other network protocols such MQTT and MOdbus TCP.  I have to say at first it was really overwhelming work to read and understand for me but perseverance sometimes pays. It's work it's called Simple Components and you can find it here

The HTTP at first was too much work for someone with too little experience in Ada, it uses gnat.Sockets and there were a lot of code to port, then I realize the MQTT client was simpler in the sense that it was less code to port so I start looking at it closer. I also notice that there is a raw lwip C implementation of MQTT which serve me as a guide when in doubt.

MQTT Client.

I did that introduction since I consider this my First attempt to port SImple Components, I actually make it work but hold to see my second attempt while doing HTTP. So keep in mind this because in the future if the project evolves MQTT might change too.

Simple Components implements an MQTT Pier Client that is simple in the sense that it's a minimal implementation, so no brokering on this version. But the author also implements the MQTT Server that supports brokering for the enthusiast :)

The first thing as explained in the docs is that all calls should be done in the context of the connection server, since I have no gnat.Sockets I end up interpreting this as to keep the Connection State for all the calls in my raw lwip server, but there is a problem, "it's all about callbacks".  Did you remember the User Data of the App layer explained in the Echo Server? Well We took advantage of it to implement the State, but moreover this time the MQTT code was on different package and MQTT_Pier type that holds the connection state was really obscure thing to port at first.

The original mqtt client package is named GNAT.Sockets.MQTT, so I start to import those files in my project and remove the GNAT.Sockets from them, so they call mqtt.adb and mqtt.ads in my project. This is mostly the MQTT stack by itself which hopefully as you will see don't have too many modifications. I implement the Client as a child package, that is Mqtt.Client package (mqtt-client.adb). It was nice to start with client since the porting was not that hard as http but the ipstack actually hasn't any client, so at the end it took me some effort but the nice part is that client is there, not as reusable as I would want it but that's another effort.

MQTT protocol handles the connection and disconnection, which is ontop of the TCP layer 4 of OSI model. So the first thing is to connect with a mqtt broker (server). For this I pick up mosquitto for my tests, it was handy since I develop under Linux and I can verify correct functioning of mqtt pier.

First attempt was to pass the MQTT_Pier type as the Udata of the TCP stack, and then kicks in kicks get out my head until I realize there are several things when working with pointers under Ada which are a little different from C/C++ world.  The short story is that MQTT_Pier is a non limited type which make me almost impossible to assign as a System.Address for later retrieval. Please don't laugh if you find an easier or better way just let me know so.

So I endup giving up trying that approach and instead keep my original array of state pool, this time I call them MSP for MQTT State Pool.

Before going further let's examine MQTT_Pier type closer.

   type MQTT_Pier
        (Max_Subscribe_Topics : Positive
        )  is new Connection with
   record
      State       : MQTT_State := MQTT_Header;
      Count       : Stream_Element_Count := 0;
      Length      : Stream_Element_Count := 0;
      Max_Size    : Stream_Element_Count := 0;
      Header      : Stream_Element       := 0;
      Version     : Stream_Element;
      Flags       : Stream_Element;
      Keep_Alive  : Duration;
      QoS         : QoS_Level;
      Packet_ID   : Packet_Identifier;
      List_Length : Natural := 0;
      --  Secondary   : Output_Buffer_Ptr;
      List        : Unbounded_Ptr_Array;
      Data        : Stream_Element_Array_Ptr :=
                       new Stream_Element_Array (1 .. Max_Message_Size);

You might wander about Connection well here it is

   type Connection is abstract new Object.Entity with
   record
      State_Pool          : Mqtt_Conn_State_Access;
      App_Cbs             : Mqtt_Client_Cbks;
      Sid                     : MS_Id;
      Client_Address    : AIP.IPaddrs.IPaddr;
      Err                      : AIP.Err_T;
   end record;

Original version has several fields on Connection record that did't apply here like socket and because I was afraid of big jumps here if you know what I mean so I end up reducing it as above, you will see later on the http port how I keep most of them except of course the socket.

The State_Pool is an access I use to attach the lwip client with mqtt code, which is needed as you will see in a moment. Sometimes is difficult to write about code without boring or even worse make the reader lost so I will pick up the example code, which the Top level code of Mqtt so I can go down to highlight the intenal mechanism of how SImple Components is connected with lwIp.

The example code serves to proof the mqtt client and as a guide for advance client code. It's shown below and you can find the source under the demos folder.

   procedure Test_1 (Client_Ip : in AIP.IPaddrs.IPaddr)
   is
      Client_Err    : AIP.Err_T;
   begin
      Set (Test_Client_Array (0), new Test_Client (Max_Subscribe_Topics => 20));
      declare
         Client   : Test_Client renames Test_Client (Ptr (Test_Client_Array (0)).all);
         PId      : Packet_Identification (Qos => At_Most_Once);
      begin
         Client.Set_Client_Ip (Client_Ip);
         Client.Set_Client_Cbs (Test_Connect'Access, Test_Receive'Access);
         Client.Connect (Client_Err);
         if Client_Err = AIP.NOERR then
            while Client.Is_Connected = False loop
               delay 0.01;
            end loop;
            Send_Connect (Client, "TestMQTTclient");
            while Client.Is_Connected = False loop
               delay 0.01;
            end loop;
            Send_Ping (Client);
            delay 1.0;
            Send_Publish (Client,
                          "makewithAda/ipstack/test",
                          "bonjour Ada world!",
                          PId);
            delay 2.0;
            Send_Disconnect (Client);
         else
            AIP.IO.Put_Line ("STM32 Connect Error");
         end if;
      end;
   end Test_1;

You specify the IP address of broker, i.e. Test_MQTT_Clients.Test_1 (Client_Ip); where Client_Ip is defined as 

Client_Ip     : constant AIP.IPaddrs.IPaddr := AIP.IPaddrs.IP4 (192, 168, 2, 5);

The first part allows to have several clients active at a given time, so you basically call new on a Test_Client and save it's reference in a Handle array.  tHe example only uses one client so later you declare a Test_Client type that renames the first entry of the Handle array. 

First thing is to apply assign the IP address of broker, perhaps the name is not the more appropriate here, anyway after that you define two callbacks one for the connection and another for the reception, both refers to TCP events, the first one might do nothing as it actually does, but the second one is very important since it was the way around I came to feed the data of TCP connection into the right MQTT Pier. I was some how lazy here since the code of reception is not correct in the sense that it pick up the first not null handle client to feed the data in. So another check should be performed, but it's so easy that let's do it right now.

First let's check the callback.

   procedure Test_Receive
     (Pcb  : AIP.PCBs.PCB_Id;
      DATA : Stream_Element_Array;
      Err  : out AIP.Err_T)
   is
   begin
      pragma Unreferenced (Pcb);
      for Hitem in Test_Client_Array'Range loop
         if Test_Client_Array (Hitem) /= Null_Handle then
            declare
               Client   : Test_Client renames Test_Client (Ptr (Test_Client_Array (Hitem)).all);
               Pointer  : Stream_Element_Offset := DATA'First;
            begin
               Client.Received (DATA, Pointer);
            end;
         end if;
      end loop;
      Err := AIP.NOERR;
   end Test_Receive;

Notice how we pass Pcb as parameter. We need to compare this with the Client Pcb, since that is private type one option is to write a little function that does for us. So let's go to mqtt.adb and add such a function we will call Does_Mqtt_Pier_Match as follows.

   function Does_Mqtt_Pier_Match
     (Pier : in out MQTT_Pier;
      Pcb   : AIP.PCBs.PCB_Id) return Boolean
   is
   begin
      return Pier.State_Pool.Pcb = Pcb;
   end Does_Mqtt_Pier_Match;

Now we can rewrite the callback as follows

   procedure Test_Receive
     (Pcb  : AIP.PCBs.PCB_Id;
      DATA : Stream_Element_Array;
      Err  : out AIP.Err_T)
   is
   begin
      for Hitem in Test_Client_Array'Range loop
         if Test_Client_Array (Hitem) /= Null_Handle then
            declare
               Client   : Test_Client renames Test_Client (Ptr (Test_Client_Array (Hitem)).all);
               Pointer  : Stream_Element_Offset := DATA'First;
            begin
               if Client.Does_Mqtt_Pier_Match (Pcb) then
                  Client.Received (DATA, Pointer);
               end if;
            end;
         end if;
      end loop;
      Err := AIP.NOERR;
   end Test_Receive;

You start getting the idea, so now let continue with the demo code. After callbacks are set we do a Connect, which indeed calls Mqtt_Client_Connect with the parameters of Client. Here is the code.

   --------------------------
   -- Mqtt_Client_Connect ---
   --------------------------

   procedure Mqtt_Client_Connect
     (Pier           : in out MQTT_Pier;
      Port           : Natural := MQTT_Port;
      Err            : out AIP.Err_T)
   is
      Pcb    : AIP.PCBs.PCB_Id;
      Sid    : MS_Id;
   begin
      --  check if client already is allocated or sort of
      if Pier.State_Pool /= null then
         Err := AIP.ERR_MEM;
         return;
      end if;

      AIP.TCP.TCP_New (Pcb);
      if Pcb = AIP.PCBs.NOPCB then
         Err := AIP.ERR_MEM;
         return;
      end if;
      AIP.TCP.TCP_Bind
        (PCB        => Pcb,
         Local_IP   => AIP.IPaddrs.IP_ADDR_ANY,
         Local_Port => 0,
         Err        => Err);
      if Err /= AIP.NOERR then
         goto Tcp_Error;
      end if;
      AIP.TCP.TCP_Connect
        (PCB        => Pcb,
         Addr       => Pier.Client_Address,
         Port       => Net.Uint16 (Port),
         Cb         => RAW_TCP_Callbacks.To_CBID (MQTT_Process_Connect'Access),
         Err        => Err);
      if Err /= AIP.NOERR then
         goto Tcp_Error;
      end if;
      MS_Alloc (Sid);
      if Sid = NOMS then
         --  AIP.TCP.TCP_Free (Pcb); --  I think original stack miss this need
         Err := AIP.ERR_MEM;
         goto Tcp_Error;
      else
         Pier.Sid := Sid;
         Pier.State_Pool := MSP (Sid)'Access;
         Pier.State_Pool.Kind := TCP_CONNECTING;
         Pier.State_Pool.Pcb := Pcb;
         Pier.State_Pool.Buf := AIP.Buffers.NOBUF;
         MSP (Sid).App_Client_Cbs := Pier.App_Cbs;
         AIP.TCP.TCP_Set_Udata (Pcb, MSP (Sid)'Address);
         --  Start cyclic timer for the corresponding client
         Timers.Timer_Alloc (Pier.State_Pool.Tmr_Id, MQTT_Timer'Access);
         Timers.Set_Interval (Pier.State_Pool.Tmr_Id, MQTT_CYCLIC_TIMER_INTERVAL * MQTT_Tick_Interval);
         Pier.State_Pool.Cyclic_Tick := 0;
      end if;
      --  set error callback
      AIP.TCP.On_TCP_Abort
        (Pcb, RAW_TCP_Callbacks.To_CBID (MQTT_Process_Abort'Access));
      return;
<<Tcp_Error>>
      declare
         tcpErr  : AIP.Err_T;
      begin
         pragma Unreferenced (tcpErr);
         AIP.TCP.TCP_Close (Pcb, tcpErr);
      end;
   end Mqtt_Client_Connect;

First part check if client is in use, as actually after the connection gets thru the pointer is set to the assigned MSP variable of Pool.  We still use the TCP_Set_Udata to save the address of it for recovering at the callbacks.

Original code from Dmitry don't has the Timers, not sure but at least the MQTT Pier doesn't have it, but after looking at the lwip implementation I decide it was necessary so the Timer is allocated and set with initial interval, some sanity check and the abort callback. The Connect callback is set in the same TCP_Connect procedure and it's code is as follow.

 

   --------------------------
   -- MQTT_Process_Connect --
   --------------------------

   procedure MQTT_Process_Connect
     (Ev  : AIP.TCP.TCP_Event_T;
      Pcb : AIP.PCBs.PCB_Id;
      Err : out AIP.Err_T)
   is
      Ms    :  Mqtt_Conn_State;
      for Ms'Address use AIP.TCP.TCP_Udata (Pcb);
      Data  : Stream_Element_Array (0 .. -1);
   begin
      pragma Unreferenced (Ev);
      --  AIP.IO.Put_Line ("New PCB Accept: Sid");
      AIP.TCP.On_TCP_Sent
        (Pcb, RAW_TCP_Callbacks.To_CBID (MQTT_Process_Sent'Access));
      AIP.TCP.On_TCP_Recv
        (Pcb, RAW_TCP_Callbacks.To_CBID (MQTT_Process_Recv'Access));
      AIP.TCP.On_TCP_Poll
        (Pcb, RAW_TCP_Callbacks.To_CBID (MQTT_Process_Poll'Access), 2 * 500);

      --  Enter MQTT connect state
      Ms.Kind := TCP_CONNECTED;
      --  Reset the Timer Tick for client
      Ms.Cyclic_Tick := 0;
      --  Call the User Procedure if available
      RAW_MQTT_Dispatcher.MQTT_Event
        (PCB    =>    Pcb,
         DATA   =>    Data,
         Cbid   =>    Ms.App_Client_Cbs.App_Connect_Cb,
         Err    =>    Err);

   end MQTT_Process_Connect;

You should be already familiar with the callbacks here except that we are calling the higher layer TCP connect callback, remember the one that does nothing. Let's move into reception to close that loop. The Reception callback is not very different from echo server, but instead of forward the Pcb to an Echo_Send, we need to dispatch the received data to upper layer, so here is the dispatching function. 

   ------------------------
   -- Mqtt_Dispatch_Recv --
   ------------------------

   procedure Mqtt_Dispatch_Recv
     (Pcb : AIP.PCBs.PCB_Id;
      Ms  : in out Mqtt_Conn_State)
   is
      Buf         : AIP.Buffers.Buffer_Id;
      srcAdd      : System.Address;
      Plen        : AIP.U16_T;
      TPlen       : AIP.U16_T;
--        Err         : AIP.Err_T := AIP.NOERR;
      Item_Size   : Stream_Element_Offset;
      Err         : AIP.Err_T;
   begin
      if Ms.Buf = AIP.Buffers.NOBUF then
         return;
      end if;
      TPlen := AIP.Buffers.Buffer_Tlen (Ms.Buf);
      declare
         Data     : Stream_Element_Array (1 .. Stream_Element_Offset (TPlen));
         Pointer  : Stream_Element_Offset := Data'First;
      begin
         loop
            Buf := Ms.Buf;
            Plen := AIP.Buffers.Buffer_Len (Buf);
            Item_Size := Stream_Element_Offset (Plen);
            declare
               type SEA_Pointer is
                 access all Stream_Element_Array (1 .. Item_Size);
               srcPtr   : SEA_Pointer;
               function As_SEA_Pointer is
                 new Ada.Unchecked_Conversion (System.Address, SEA_Pointer);
               Data_Ptr : SEA_Pointer;
            begin
               srcAdd := AIP.Buffers.Buffer_Payload (Buf);
               srcPtr := As_SEA_Pointer (srcAdd);
               Data_Ptr := As_SEA_Pointer (Data (Pointer)'Address);
               Data_Ptr.all (1 .. Item_Size) := srcPtr.all (1 .. Item_Size);
               --  Grab reference to the following Buf, if any
               Ms.Buf := AIP.Buffers.Buffer_Next (Buf);
               if Ms.Buf /= AIP.Buffers.NOBUF then
                  AIP.Buffers.Buffer_Ref (Ms.Buf);
               end if;
               --  Deallocate the processed buffer
               AIP.Buffers.Buffer_Blind_Free (Buf);
               
               exit when Ms.Buf = AIP.Buffers.NOBUF;
               --  Pointer "points" to Next element to copy
               Pointer := Pointer + Item_Size + 1;
               if Pointer > Data'Last then
                  Raise_Exception
                    (Layout_Error'Identity,
                     "Invalid pointer"
                    );
               end if;
            end;
         end loop;
         --  Signal TCP layer that we can accept more data
         AIP.TCP.TCP_Recved (Pcb, TPlen);
         pragma Warnings (Off, """Err"" modified by call, *");
         RAW_MQTT_Dispatcher.MQTT_Event
           (PCB    =>    Pcb,
            DATA   =>    Data,
            Cbid   =>    Ms.App_Client_Cbs.App_Receive_Cb,
            Err    =>    Err);
         pragma Warnings (On, """Err"" modified by call, *");
      end;
   end Mqtt_Dispatch_Recv;

This was the part of the project that I learn (one way good or bad?) how to write to a Stream_Element_Array. At the end of the loop Data ends up with the received bytes, so the callback is done.

Mqtt Client Send.

The Mqtt stack reacts to events, the "main" code initiates a TCP connect, then it follows the Mqtt Connect Command which obvioulsy need to send a TCP segment, so we create the Send procedure that will be used all over the mqtt package when there is an Stream_Element_Array of data to send. The code lives in the mqtt.adb and is shown below.

   procedure Send
             (Pier : in out MQTT_Pier;
              Data : Stream_Element_Array
             ) is
      Pointer     : Stream_Element_Offset := Data'First;
      size        : AIP.Buffers.Data_Length;
      BufLen      : AIP.Buffers.Buffer_Length;
      p           : AIP.Buffers.Buffer_Id;
      dstAdd      : System.Address;
      q           : AIP.Buffers.Buffer_Id;
      Item_Size   : Stream_Element_Offset;
   begin
      size := AIP.U16_T (Data'Last);
      AIP.Buffers.Buffer_Alloc (0, size, AIP.Buffers.LINK_BUF, p);
      if p /= AIP.Buffers.NOBUF then
         q := p;
         loop
            BufLen := AIP.Buffers.Buffer_Len (q);
            Item_Size := Stream_Element_Offset (BufLen);
            declare
               type SEA_Pointer is
                 access all Stream_Element_Array (1 .. Item_Size);
               dstPtr   : SEA_Pointer;
               function As_SEA_Pointer is
                 new Ada.Unchecked_Conversion (System.Address, SEA_Pointer);
               Data_Ptr : SEA_Pointer;
            begin
               dstAdd := AIP.Buffers.Buffer_Payload (q);
               dstPtr := As_SEA_Pointer (dstAdd);
               Data_Ptr := As_SEA_Pointer (Data (Pointer)'Address);
               --  copy the actual data using the streams access pointers
               dstPtr.all (1 .. Item_Size) := Data_Ptr.all (1 .. Item_Size);
            end;
            q := AIP.Buffers.Buffer_Next (q);
            --  check if there is one more buffer to fill
            exit when q = AIP.Buffers.NOBUF;
            --  Pointer "points" to Next element to copy
            Pointer := Pointer + Item_Size + 1;
            if Pointer > Data'Last then
               Raise_Exception
                 (Layout_Error'Identity,
                  "Invalid pointer"
                 );
            end if;
         end loop;
         declare
            Ms : Mqtt_Conn_State;
            for Ms'Address use Clients.MSP (Pier.Sid)'Address;
         begin
            Ms.Buf := p;
            Clients.Mqtt_Send (Ms.Pcb, Ms);
         end;
      else
         Pier.Err := ERR_MEM;
      end if;
   end Send;

Somehow is the inverse operation of reception, this time we need to reserve lwip buffers to hold the data. 

Mqtt Timer.

Last but not least the mqtt timer helps with Keep alive functionality, for more info go here. The keepalive is implemented for each connected client and the code is below.

   -----------------------------------------------
   --  MQTT_Timer --  CycleTick Timer Callback  --
   -----------------------------------------------

   procedure MQTT_Timer (Id : Integer) with
     Refined_Global => (In_Out => (MSP))
   is
      Sid   :  MS_Id := NOMS;
   begin
      null;
      for J in MSP'Range loop
         if MSP (J).Tmr_Id = Id then
            Sid := J;
            exit;
         end if;
      end loop;
      if Sid = NOMS then
         return;     --  this should not happen
      end if;
      --  Sid Points to the Client Connection State Data that MQTT_Timer belongs
      case MSP (Sid).Kind is
         when MS_CONNECTING =>
            MSP (Sid).Cyclic_Tick := MSP (Sid).Cyclic_Tick + 1;
            if MSP (Sid).Cyclic_Tick * MQTT_CYCLIC_TIMER_INTERVAL >= MQTT_CONNECT_TIMOUT then
               --  Disconnect TCP
               Mqtt_Close (MSP (Sid).Pcb, MSP (Sid));
               Timers.Timer_Stop (Id);
            end if;
         when MS_CONNECTED =>
            --  keep_alive > 0 means keep alive functionality shall be used
            if MSP (Sid).Keep_Alive > 0 then
               MSP (Sid).Server_Watchdog := MSP (Sid).Server_Watchdog + 1;
               --  If reception from server has been idle for 1.5*keep_alive time,
               --  server is considered unresponsive
               if MSP (Sid).Server_Watchdog * MQTT_CYCLIC_TIMER_INTERVAL >=
                 MSP (Sid).Keep_Alive + MSP (Sid).Keep_Alive / 2
               then
                  Mqtt_Close (MSP (Sid).Pcb, MSP (Sid));
                  Timers.Timer_Stop (Id);
               end if;
               if MSP (Sid).Cyclic_Tick * MQTT_CYCLIC_TIMER_INTERVAL >=
                 MSP (Sid).Keep_Alive
               then
                  --  Sending keep-alive message to server
                  declare
                     Ps : MQTT_Pier_Ptr;
                     for Ps'Address use AIP.TCP.TCP_Udata (MSP (Sid).Pcb);
                  begin
                     Send_Ping (Ps.all);
                     MSP (Sid).Cyclic_Tick := 0;
                  end;
               else
                  MSP (Sid).Cyclic_Tick := MSP (Sid).Cyclic_Tick + 1;
               end if;
            end if;
         when TCP_CONNECTING =>
            MSP (Sid).Cyclic_Tick := MSP (Sid).Cyclic_Tick + 1;
            if MSP (Sid).Cyclic_Tick * MQTT_CYCLIC_TIMER_INTERVAL >= TCP_CONNECT_TIMOUT then
               --  Disconnect TCP
               Mqtt_Close (MSP (Sid).Pcb, MSP (Sid));
               Timers.Timer_Stop (Id);
            end if;
         when others =>
            --  Timer should not be running in this state - perhaps TCP_ABORT
            Timers.Timer_Stop (Id);
      end case;
   end MQTT_Timer;

Notice that it might close and unresponsive connection and also send the Ping Commands when needed by keep-alive.

Demo Time. 

A short video of the Test demo code follows.

HTTP Server

With more knowledge and previous experience I toward to HTTP server porting. The HTTP require the connection state machine and it's http state machine which heavily depend on gnat.Sockets.Server which fortunately is a package under the Simple Components source. This Server package has a task worker that handle all incoming connections using gnat.sockets, so a big part of the code doesn't apply here to the lwip scheme, but some supporting code was needed in its place, so what I did was to copy all of it into a rename package called RAW_LwIp_Server (raw_lwip_server.adb) and star commenting code like if I wasn't sure what I was doing, yes very crazy moment just desperate to get rid of all compiler errors and warnings.

At the end I used like 60 % of the code but the commented original code is still there since I haven't tested all the HTTP code, so basically I have a working Hello World .htm generated in code. It was my desire to be able to load .htm from the SD card but actually I had a hard time to make the SD card file system work on my  STM32F769I_DISCO.

Let's discuss how it works and have a final demo (yes you can go ahead to watch the demo first! ;)

It's difficult and boring to step over each detail here, so similar to mqtt explanation, I will start from the Demo Code and try to give as much details as possible.

Hello World.

The demo consists of the classic hello world HTTP GET. The package Test_HTTP_Servers contains all the code, from the "main" file we only start the task which is suspended waiting for a flag. The task called worker is shown below.

   --  ------------------------------
   --  Start the HTTP Server Loop.
   --  ------------------------------
   procedure Start is
   begin
      Ada.Synchronous_Task_Control.Set_True (HTTP_Ready);
   end Start;

   task body Worker is

      timer_deadline : Ada.Real_Time.Time;

   begin
      --  Wait until started
      Ada.Synchronous_Task_Control.Suspend_Until_True (HTTP_Ready);
      AIP.IO.Put_Line ("HTTP Server Task Starting");
      declare
         Factory : aliased Test_HTTP_Factory
           (Request_Length  => 200,
            Input_Size      => 1024,
            Output_Size     => 1024,
            Max_Connections => 100
           );
         Server : Connections_Server (Factory'Access, Port_Type (80));
      begin
         timer_deadline := Ada.Real_Time.Clock;
         loop
            Server.Poll_Connections;
            delay until timer_deadline;
            timer_deadline := timer_deadline + HTTP_Polling;
         end loop;
      end;

   end Worker;

Factory and Server types were the reasons to have the original Server ported, they provide an easy and clean way to create the Server.  The Connections_Factory is redefined as

   type Connections_Factory is
     new Ada.Finalization.Limited_Controlled with
      record
         Port          : Natural;
         Socket_SP     : Socket_State_Access_Array;
      end record;

where similar to the State Pool seen before, Socket_State_Access_Array is an array of those Pools.

   type Socket_State is record
      Kind              : State_Kind;
      Pcb               : AIP.PCBs.PCB_Id;
      Buf                : AIP.Buffers.Buffer_Id;
      Err                : AIP.Err_T;
      Server_Cbs    : Socket_Server_Cbks;
      Client            : Connection_Ptr;
   end record;
   type Socket_State_Access is access all Socket_State;
   type Socket_State_Array is array (Valid_SS_Id) of aliased Socket_State;
   type Socket_State_Access_Array is array (Valid_SS_Id) of Socket_State_Access;

instead of having an assignment of State_Pool pointers, here we have a one to one relationship of the pointer array with the serve pool array. This is independent of HTTP code, so other server can use the same Factory primitive right now.

Connections_Server  has been reduce to

   type Connections_Server
     (Factory : access Connections_Factory'Class;
      Port    : Port_Type
     )  is new Ada.Finalization.Limited_Controlled with
      record
         Clients          : Natural := 0;
         Servers          : Natural := 0;
         Postponed_Count  : Natural := 0;
         Postponed        : Connection_Ptr;
         Unblock_Send     : Boolean  := False;
      end record;

The trick here is that when you instantiate a Connections_Server it will call Initialize, as my knowledge holds this is because it's a type of Ada.Finalization.Limited_Controlled, so an Initialize procedure will override the default one, which is the case of 

   overriding
   procedure Initialize (Listener : in out Connections_Server) is
      Err : AIP.Err_T;
   begin
--        pragma Unreferenced (Err, Listener);
      Listener.Factory.Port := Integer (Listener.Port);
      API_Process.Initialize (Listener'Access, Listener.Factory, Err);
      if Err /= AIP.NOERR then
         AIP.IO.Put_Line ("Error Init Server");
      end if;
   end Initialize;

In order to not mess up the original Server file, I created a child package with the underlying already well known ipstack server, called API_Process. The structure is very similar to a mix of the Echo_Server seen before and the MQTT_Client since we use similar dispatch procedure for receiving data, but there are some novel changes here of course, let's discuss them. (Notice this is general Server that can be used for other protocol servers)

Initialize.

Let's examine the Initialize code shown below.

   ----------
   -- Init --
   ----------

   procedure Initialize
     (Listener       : access Connections_Server'Class;
      Factory        : access Connections_Factory'Class;
      Err            : out AIP.Err_T)
   is
      Pcb            : AIP.PCBs.PCB_Id;
      Port           : Integer;
   begin
--        pragma Unreferenced (Pcb, Port, Err);
--        pragma Unreferenced (Listener);

      Init_SS_Pool;
      lwIp_Listener := Listener.all'Unchecked_Access;
      lwIp_Factory := Factory.all'Unchecked_Access;

      Port := Integer (lwIp_Factory.Port);

      AIP.TCP.TCP_New (Pcb);
      if Pcb = AIP.PCBs.NOPCB then
         Err := AIP.ERR_MEM;
      else
         AIP.TCP.TCP_Bind
           (PCB        => Pcb,
            Local_IP   => AIP.IPaddrs.IP_ADDR_ANY,
            Local_Port => AIP.U16_T (Port),
            Err        => Err);
      end if;

      if Err = AIP.NOERR then
         AIP.TCP.TCP_Listen (Pcb, Err);
         pragma Assert (AIP.No (Err));
         --  Factory Socket State Pool is linked
         for Sid in lwIp_Factory.Socket_SP'Range loop
            lwIp_Factory.Socket_SP (Sid) := SSP (Sid)'Access;
         end loop;
         AIP.TCP.On_TCP_Accept
           (Pcb, RAW_TCP_Callbacks.To_CBID (SOCKET_Process_Accept'Access));
      end if;

   end Initialize;

The problem once again was how to link the lwip state pool with the application layer. The answer after a huge fight against my ignorance in Ada language was to have a pointer to the Connections_Server and Factory, right now the code only allows one server since there are only one instance of each, so further adjustment is needed to have two servers.

The access types are declared as 

   lwIp_Listener   : Connections_Server_Ptr;
   lwIp_Factory    : access Connections_Factory'Class;

and you can see how the assignment is done, after the server is bonded and listening, the state pool is assigned as well, this time indexed by the Sid. Finally the Accept callback is configured. The Accept callback has new stuff to discover

   procedure SOCKET_Process_Accept
     (Ev  : AIP.TCP.TCP_Event_T;
      Pcb : AIP.PCBs.PCB_Id;
      Err : out AIP.Err_T)
   is
--        Factory  : constant access Connections_Factory'Class := ;
      Sid      : SS_Id;
      Factory : access Connections_Factory'Class renames lwIp_Factory;
   begin
      Err := AIP.NOERR;
      SS_Alloc (Sid);

      if Sid = NOSS then
         Err := AIP.ERR_MEM;
      else
         Factory.Socket_SP (Sid).Kind := SS_ACCEPTED;
         Factory.Socket_SP (Sid).Pcb := Pcb;
         Factory.Socket_SP (Sid).Buf := AIP.Buffers.NOBUF;

         AIP.TCP.TCP_Set_Udata (Pcb, SSP (Sid)'Address);

         AIP.TCP.On_TCP_Sent
           (Pcb, RAW_TCP_Callbacks.To_CBID (SOCKET_Process_Sent'Access));
         AIP.TCP.On_TCP_Recv
           (Pcb, RAW_TCP_Callbacks.To_CBID (SOCKET_Process_Recv'Access));
         AIP.TCP.On_TCP_Abort
           (Pcb, RAW_TCP_Callbacks.To_CBID (SOCKET_Process_Abort'Access));
         AIP.TCP.On_TCP_Poll
           (Pcb, RAW_TCP_Callbacks.To_CBID (SOCKET_Process_Poll'Access), 500);

         AIP.TCP.TCP_Accepted (Pcb);

         declare
            Data : Ada.Streams.Stream_Element_Array (1 .. 1);
         begin
            --  This is our way around to associate the Server Connection
            --  Pool to a given Pcb and thou fill specific client callbacks
            --  at the upper layer.
            Data (Data'First) := Ada.Streams.Stream_Element (Sid);
            RAW_SOCKET_Dispatcher.SOCKET_Event
              (Client  =>    Factory.Socket_SP (Sid).Client,
               PCB     =>    Pcb,
               DATA    =>    Data,  --  Not really used in upper layer let's see...
               Cbid    =>    RAW_SOCKET_Callbacks.To_CBID (Do_Create'Access),
               Err     =>    Err);
            if Factory.Socket_SP (Sid).Client = null then
               Err := AIP.ERR_MEM;
               Socket_Close (Pcb, SSP (Sid));
            else
               declare
                  This : Connection'Class renames Factory.Socket_SP (Sid).Client.all;
               begin
                  This.Sid            := Sid;
                  This.Client         := False;
                  This.Connect_No     := 0;
                  This.Client_Address := Ev.Addr;
                  This.Client_Port    := Ev.Port;
                  This.Err            := AIP.NOERR;
                  Clear (This);
                  This.Listener := lwIp_Listener.all'Unchecked_Access;
                  if This.Transport = null then -- Ready
                     This.Session := Session_Connected;
                     Connected (This);
                  else
                     This.Session := Session_Handshaking;
                  end if;
                  Err := This.Err;
               end;
            end if;
         end;
      end if;
   end SOCKET_Process_Accept;

If Socket State allocation goes well we set the callbacks and accept the connection as before, also we modify the State Pool, but notice we use the access pointer of our factory from now on. Thereafter comes a callback to the parent Server package, this callback is a replacement of the underlying overriding behavior of gnat.Sockets that original code used to take advantage of, so the callback is actually calling the Create procedure in RAW_lwip_server shown below.

   procedure Do_Create
     (Client  : in out Connection_Ptr;
      Pcb     : AIP.PCBs.PCB_Id;
      DATA    : Stream_Element_Array;
      Err     : out AIP.Err_T)
   is
   begin
      Client := Create
        (Factory  =>  API_Process.lwIp_Factory,  --  API_Process.Server'Access,
         Pcb      =>  Pcb,
         Data     =>  DATA,
         Err      =>  Err);
   end Do_Create;


   function Create
     (Factory  : access Connections_Factory;
      Pcb  : AIP.PCBs.PCB_Id;
      Data : Stream_Element_Array;
      Err  : out AIP.Err_T
     ) return Connection_Ptr is
   begin
      pragma Unreferenced (Factory, Pcb, Data, Err);
      --  Default Implementation returns null.
      return null;
   end Create;

As you can see by default it does nothing, but the overriding Create procedure in Test_HTTP_Servers executes in place of

   overriding
   function Create
     (Factory  : access Test_HTTP_Factory;
      Pcb      : AIP.PCBs.PCB_Id;
      Data     : Stream_Element_Array;
      Err      : out AIP.Err_T
     )  return Connection_Ptr is
      Result : Connection_Ptr;
   begin
      pragma Unreferenced (Pcb, Data);
      --        if Get_Clients_Count (Listener.all) < Factory.Max_Connections then
      Result :=
        new Test_HTTP_Client
          (Request_Length   => Factory.Request_Length,
           Input_Size     => Factory.Input_Size,
           Output_Size    => Factory.Output_Size
          );
      --     Receive_Body_Tracing   (Test_Client (Result.all), True);
      --     Receive_Header_Tracing (Test_Client (Result.all), True);
      Err := AIP.NOERR;
      return Result;
   end Create;

It's under this Create function that HTTP Clients are created and final bonding to the Pcb is done when returns to Accept callback of lwip. Notice that we check the Client for null value, in such a case it fails to create it we proceed to close the connection, otherwise an initialization is done where Ip and Port number are set from the Event received.

The receive dispatch procedure is very similar, will only show the callback in this case. Notice that it's fixed to always call a  Do_Receive. 

 RAW_SOCKET_Dispatcher.SOCKET_Event
   (Client =>    Ss.Client,
    PCB    =>    Pcb,
    DATA   =>    Data,
    Cbid   =>    RAW_SOCKET_Callbacks.To_CBID (Do_Receive'Access),
    Err    =>    Err);

Do_Receive and the Received are similar to Do_Create, this time the Received procedure is overwritting in Connections_State_Machine.HTTP_Servers package, since this has the ability to dispatch the request accordingly. The Test Server implements higher level of abstraction, for example the hello world demo is responded using Do_Get procedure, here is for reference

   overriding
   procedure Do_Get (Client : in out Test_HTTP_Client)
   is
      Status : Status_Line renames Get_Status_Line (Client);
   begin
      case Status.Kind is
         when None =>
            null;
         when File =>
            if Status.File = "hello.htm" then
               Send_Status_Line (Client, 200, "OK");     -- Response status line
               Send_Date   (Client);                     -- Date header line
               Send_Server (Client);                     -- Server name
               Send_Connection (Client, False);
               Send_Content_Type (Client, "text/html");  -- Content type
               Accumulate_Body (Client, "<html><body>"); -- Begin content construction
               Accumulate_Body (Client, "<p>Hello world!</p>");
               Accumulate_Body (Client, "</body></html>");
               Send_Body (Client);
--                 Send_Body_Now (Client);                  -- Evaluate total length, send length
            end if;
         when URI =>
            null;
      end case;
   end Do_Get;

We are not ready for the demo yet. Remember the HTTP task created? Original code uses gnat.Sockets to pool network descriptors like a select will do on sockets, ready for read, ready for write and so.. it was very cleaver how it works when I finally figure it out but actually does not apply here, but since we need to adapt our code, it happens that the HTTP state machine has a funny way to work. Each HTTP client has a type that is very long record defined at HTTP_Server but important to notice is that is a new type of type State_Machine which in fact is new type of Connection, the one that we mess up during MQTT client.  But now we are using Connection type as original code (which means MQTT Client can be improved and bring broker alive one of these days...).

 I was going to highlight that Connection type has a Read and Written Buffer which the HTTP (and others under Simple Components) server read and write to send data over the network, and with some cleaver logic application informs the socket task (Worker) when there is Data to send, in the Simple Components documentation this is referred as to Unblock. Application calls like before example code, such as Send_Status_Line (Client, 200, "OK");  write to the Written buffer and then Unblock the "socket" for sending it. 

I was going to give up when I realize it but then I thought ipstack server can also has a task that uses "almost" similar mechanism to unblock the "Send".  So I present to you the Pool_Connections procedure that mimic such original code or sort of it.

   procedure Poll_Connections (Listener  : in out Connections_Server) is
      fab   : access Connections_Factory'Class;
      Block : Boolean;
   begin
      while Listener.Unblock_Send loop
         Listener.Unblock_Send := False;
         fab := Listener.Factory.all'Unchecked_Access;
         for Sid in fab.Socket_SP'Range loop
            if fab.Socket_SP (Sid).Kind = SS_RECEIVED then
               declare
                  Client : Connection_Ptr renames fab.Socket_SP (Sid).Client;
               begin
                  if not Client.all.Written.Send_Blocked then
                     --  Time to Process this Client Buffer
                     Write (Client.all, Block);
                     if not Block then
                        Listener.Unblock_Send := True;
                     else
                        Client.all.Written.Send_Blocked := True;
                        if Client.all.Data_Sent then
                           Data_Sent (Listener, Client);
                        end if;
                     end if;
                  end if;
               end;
            end if;
         end loop;
      end loop;
   end Poll_Connections;

I setup this procedure to be call from Worker task every few milliseconds, haven't play a lot with it but it works well for 50 or 20 ms. Here you can see a tcpdump capture of the hello.htm GET request (please notice the  ARP cache was outdated during this capture).

Demo Video of HTTP Hello World.

Bogus Bonus.

I did my best to do something extra during the deadline extension of three days, so I end up porting the DNS code from ada-enet to the ipstack. The example is included and working capture of google lookup is show here.

The last dns code is already on github (below the link)

The code uses a similar approach to ada-enet, it defines an array of Query, but instead of the UDP socket from ada-enet, I uses a connection object as shown below. This approach uses a private Array of Dns queries access to points to the Application Array defined in Dns_List package. 


   type UdpClient is abstract new Object.Entity with
   record
      Udp_State_Pool   : Dns_Conn_State_Access;
      Sid              : DS_Id;
      Client_Address   : AIP.IPaddrs.IPaddr;
      Err              : AIP.Err_T;
   end record;

   protected type Request is
      procedure Set_Result (Addr : in AIP.IPaddrs.IPaddr;
                            Time : in Net.Uint32);
      procedure Set_Status (State : in Status_Type);
      function Get_IP return AIP.IPaddrs.IPaddr;
      function Get_Status return Status_Type;
      function Get_TTL return Net.Uint32;
   private
      Status   : Status_Type := NOQUERY;
      Ip       : AIP.IPaddrs.IPaddr := 0;
      Ttl      : Net.Uint32;
   end Request;

   type Query is new UdpClient with record
      Name     : String (1 .. DNS_NAME_MAX_LENGTH);
      Name_Len : Natural := 0;
      Deadline : Ada.Real_Time.Time;
      Xid      : Net.Uint16;
      Result   : Request;
   end record;

The function DNS.Init has to be call before any Resolve, the initialization pass the Array of Query to the package DNS in order to set the access pointers correctly. 

   procedure Init (Query_Array : access Dns_Query_Array)
   is
   begin
      for item in Dns_Queries'Range loop
         Dns_Queries (item) := Query_Array (item)'Access;
      end loop;
      Init_Done := True;
   end Init;

After that a Resolve can be done very similar to original ada-enet. The example code shows a simple loop to wait until DNS server respond. Below is the code.

      Dns_List.Queries (1).Resolve ("www.google.com");      
      --  This won't get thru since original code accept non-authoritative only
--        Dns_List.Queries (1).Resolve ("x83vdeb2.localdeb.net");
      declare
         Timeout : Natural := 0;
      begin
         timer_deadline := Ada.Real_Time.Clock;
         loop
            timer_deadline := timer_deadline + timer_delay;
            delay until timer_deadline;
            if Dns_List.Queries (1).Get_Status = NOERROR then
               if Dns_List.Queries (1).Get_Name'Length > 0 then
                  AIP.IO.Put_Line (Dns_List.Queries (1).Get_Name);
                  AIP.IO.Put_Line (To_String (Dns_List.Queries (1).Get_Ip));
                  exit;
               end if;
            elsif Timeout > 10 then
               AIP.IO.Put_Line ("DNS Lookup Timeout");
               exit;
            else
               Timeout := Timeout + 1;
            end if;   
         end loop;
      end;

I intentionally left a comment in the code above about the local query, this is because I setup dnsmasq on my development machine to work on the interface I have connected the discovery board. I found that there is a check performed by ada-enet code and Flags received, in this case the Flags value was 0x8580, and the code checks for 0x8100, this is because  the domain is local the NDS Answer as an Authoritative Server, setting that bit which makes it 0x85xx in this case.

 I haven't study the code to know if it can parse or not the AnswerThe important oart is that  there is one less thing to work on for a more complete ipstack (it remains DHCP and NTP which ada-enet provides).

Source code and build instructions can be found on github project so the idea is to call for contributors and testers.

Known Issues.

There should be a section on git project to talk about this, but let's mention here for project completeness. 

As I have commented, the lwIP was modified, I did my best with a lot of pressure to get results in doing so, but more tests are needed and possible some rework. 

One issue I have notice is about some packet from HTTP server after some time during operation and several queries that seems to break state machine or client state. The tcpdump.zip file contains some captures that contains unexpected segments, for example capture FIN_ACK_RETRANS shows how several HTTP GET queries were successfully handle by server but then unexpected FIN-ACK re transmission appears. 

Similar unexpected segment from server was shown in another capture, this time a RST-ACK segment during what seems to be a normal GET HTTP query. 

Also the captures show coincidental or not with the unexpected segments how the Server send some ICMP segments, I notice it and comment the part of the code that executes those Pings, original ipstack IP_Input procedure while dispatching packets, if it doesn't find the  protocol the packet belongs it send's an ICMP_Reject packet (See Dispatch_Upper procedure inside IP_Input). That might not be the case here but also a couple of packets from my Linux machine came in just before with MAC address of broadcast, perhaps the ARP engine is failing here.

The reason I add a permanent entry to the STM ARP cache was because I notice it fails to subsequent Server responses when ARP was in he middle, it's necessary more tests here since at the end I also correct a problem with pbufs been allocated and not free which might have to do with ARP since it uses a lot the Unack_Queue when working this way.

It happen that buffers are allocated using TCP_New, but somehow during development I break the mechanism of TCP Queues, actually I found an issue that makes me go back to test the ipstack on my Linux host to confirm if the issue was there, and it actually has a bug, I suspect is related to the buffer allocation and queues since I correct it for the stm32 ipstack, it happen that if you take a buffer out of some Queue you better do something with the buffer after that. For example TCP_Output get buffers from the Send_Queue and after sending the segments it place the buffer in the Unack_Queue so the logic that follows is that after they are taken out from Unack_Queue they should be Free, otherwise they will live in memory, so the problem will not be notice until TCP_New fails which will brake the state machine. Which was actually sort of the issue I notice during ARP queries.

You can easily confirm the ipstack bug, it took me some iteration to make it work under linux since the low level driver does something with OS calls to configure the interface, so I manually tweak the stuff to make it work. Just briefly, there is a config file of ipstack called aip-config.ads, the number of data buffer statically allocated is Data_Buffer_Num which is 32, just do the test an lower it to 5 for example in order to test it quickly, then using the echo server try to do five "echo tests", that is, connect, send some data and disconnect, for sure it will end up with an exception and program ends abruptly. I found the issue since the embedded was doing the same, then after a long debugging session I notice that the Unack_Queue state when I new connection starts was holding an always increasing buffer_id number, which should not happen since disconnect should free the memory.

I mentioned somewhere that Callbacks from Timer and in general callbacks from the TCP stack should not take too much before returning in, the MQTT_DELAY_EXAMPLE pcap capture shows the effect of a delay, you can notice the FIN-ACK from server that waits for the Last_ack, in this case from our MQTT client is re-transmitted, but can notice that the ACK is then transmitted, after debugging the issue I notice it was the semihosting printing on the disconnect callback. As you know semihosting here is really bad since ST-Link has not good support for it. 

Final thoughts.

This project address several problems or better said, several missing components of Ada Library, and left a good path continue development since it's 100% Ada code that can be improved and extended but also it's innovative in the way that apart from solving those lacking areas it merges different projects in a cleaver way, which due to it's nature was difficult to test and debug in the Time Frame of the Contest, so I am satisfy with results achieved.  

I couldn't discuss all the details here about the implementation but try to cover important topics, so I left aside some aspect about the porting to stm32, for example there is a folder called build under sources that has some code to handle the network packets, it happens that this was generated using xmlada which is not available for ARM version, so the trick was to include the already generated files. HTTP has some package dependency, which is Ada.Calendar that it's not available, I did my best to fill the skeleton functions but those actually do not process anything right now. I share the code in github since I believe in collaboration and free software, after all I got this opportunity in part from existing free code out there, so this will be my return and of course my willing is to have a chance in the contest to win with this project.

This project was a very good challenge to me since I have to learn Ada as quickly as possible, and not been very use to lwIP it was almost new stuff everyday to learn before starting getting some result, but the deadline pressure sometimes give you what is needed to do it, so I enjoy doing it and I appreciate the people interacted with me, specially thanks to Fabien that share me the ipstack code (it's some obscure path on git far away from google sight) and Stephane for the git issue support. 

But also specially thanks to my wife for the support on many overnight working hours on this project.


Project Files