Thursday, April 28, 2016

YellowJacket Arduino with WiShield Wifi (Part 21)

Investigating DHCP on YellowJacket WiServer —
What is DHCP?
https://en.wikipedia.org/wiki/Dynamic_Host_Configuration_Protocol explains: “The Dynamic Host Configuration Protocol (DHCP) is a standardized network protocol used on Internet Protocol (IP) networks for dynamically distributing network configuration parameters, such as IP addresses for interfaces and services.
… Most residential network routers receive a globally unique IP address within the provider network. Within a local network, DHCP assigns a local IP address to each device connected to the local network.
… When a computer or other networked device connects to a network, the DHCP client software sends a broadcast query requesting necessary information. The DHCP server manages a pool of IP addresses and information about client configuration parameters such as default gateway, domain name, the name servers, and time servers. On receiving a request, the server may respond with specific information for each client, as previously configured by an administrator, or with a specific address and any other information valid for the entire network and for the time period for which the allocation (lease) is valid.”
WiShield V1.4 includes code and an example sketch for DHCP. The Examples:SocketAppDHCP.ino sketch was modified to communicate over wifi with a Mac on the local network. The sketch requires APP_SOCKAPP, APP_UDPAPP, and UIP_DHCP to be selected in apps-conf.h as noted in the sketch comments. Initially, the sketch did not connect with the Mac. As noted in comments in the YJ57 sketch code, Wireshark was used to figure out the ip/port issues. Changing the destination port, “uint16_t dest_Port = 80;”, from 6000 to 80 in the YJ57 sketch, allowed the two devices to communicate using TCP.
To use the python script below, copy the code to a file “SocketAppDHCP_V14_02.py”, and execute that file in Terminal:
Mac: me$ python SocketAppDHCP_V14_02.py
In the SocketAppDHCP_V14_02.ino sketch, change the destination port to “uint16_t dest_Port = 3333;” which is the port being used by the python script. As the SocketAppDHCP_V14_02 sketch executes, the Mac Terminal app will display the messages sent from YJ57 to python, which include the YJ57 ip/port and “char packet[] = “SocketAppDHCP”;”:
Mac: me$ python SocketAppDHCP_V14_02.py
Connected by ('10.0.1.57', 1026)
SocketAppDHCP
Following is the SocketAppDHCP_V14_02 sketch modified for use with python script port 3333:
/*
 * SocketAppDHCP_V14_02 sketch
 * 2016-04-22
 * 84Park
 *
 * A simple socket application / DHCP example sketch for the WiShield
 * Sample python server script for this sketch to connect with at bottom of sketch
 *
 * To begin, the sketch will run as written without the python script and
 * will connect to a Mac on the local network.
 *
 * Compiled with WiShield_V14 library (ref: github.com/hamityanik/WiShield_user_contrib) and Arduino 1.6.5
 *  This code works 2016-04-22 with YJ57 board with/without SD Card Shield
 *  Pre-calculated security_data[] inits in 6 seconds
 *
 * Example Output to Serial Monitor
 *
=== Initializing WiShield as Client to display DHCP Query on Serial Monitor ===

WiFi Init OK; Start the DHCP query...
DHCP IP     : 10.0.1.57 // YJ57 Arduino IP rather than local_ip[]
DHCP GATEWAY: 10.0.1.1
DHCP NETMASK: 255.255.255.0
DHCP DNS    : 10.0.1.1
SA: connected / send
SA: newdata
SA: acked
SA: closed / timedout
 *
 */

// Requires APP_SOCKAPP, APP_UDPAPP and UIP_DHCP to be defined in apps-conf.h
//  APP_SOCKAPP  - for the TCP sockets components of the sketch
//  APP_UDPAPP   - for the UDP/DNS components of the sketch
//  UIP_DHCP     - for the DHCP components of the sketch

#include <WiShield.h>
extern "C" {
#include "uip.h"
}

// Wireless configuration parameters ----------------------------------------
unsigned char local_ip[]    = {10, 0, 1, 57};   // IP address of WiShield
unsigned char gateway_ip[]  = {10, 0, 1, 1};   // router or gateway IP address
unsigned char subnet_mask[] = {255, 255, 255, 0}; // subnet mask for the local network
char ssid[]                 = {"Network"};   // max 32 bytes
uint16_t localPort = 1026;      // local port to listen on

// The resolved DNS address
uint16_t dns_ip[]           = {10, 0, 1, 1};   // DNS server addr // typedef uint16_t u16_t;
//uint16_t dnsAddr[] = {0, 0, 0, 0}; // witypes.h typedef unsigned char uint16;  unsigned char == uint16_t;   16 bits
//char dest_URL[]  = {"w1.weather.gov"};
uint16_t dest_ip[]    = {10, 0, 1, 64};   // IP address of Mac on local network
// uint16_t dest_Port = 80;      // destination port
uint16_t dest_Port = 3333;  // destination port for use with python script below

unsigned char security_type = 5;  // 0 - open; 1 - WEP; 2 - WPA; 3 - WPA2; 4 - WPA Precalc; 5 - WPA2 Precalc

// Depending on your security_type, uncomment the appropriate type of security_data
// 0 - None (open)
// const char security_data[] PROGMEM = {};

// 1 - WEP
// UIP_WEP_KEY_LEN. 5 bytes for 64-bit key, 13 bytes for 128-bit key
// Only supply the appropriate key, do not specify 4 keys and then try to specify which to use
//const char security_data[] PROGMEM = { 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, };

// 2, 3 - WPA/WPA2 Passphrase
// 8 to 63 characters which will be used to generate the 32 byte calculated key
// Expect the g2100 to take 30 seconds to calculate the key from a passphrase
// const char security_data[] PROGMEM = {"Network_Passcode"};

// 4, 5 - WPA/WPA2 Precalc
// The 32 byte precalculate WPA/WPA2 key. This can be calculated in advance to save boot time
//const prog_char security_data[] PROGMEM = {
//    0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f,
//    0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1f, 0x1f,
//};
// Calc the Precalc:
// http://jorisvr.nl/wpapsk.html
// The derived key will appear in the form as a sequence of 64 hexadecimal digits.
/*0599967ce21000026c2cf439ea561c3d37abbf48085738cbe9b9a9ca093a6adb*/
const char security_data[] PROGMEM = {
  0x05, 0x99, 0x96, 0x7c, 0xe2, 0x10, 0x00, 0x02, 0x6c, 0x2c, 0xf4, 0x39, 0xea, 0x56, 0x1c, 0x3d,
  0x37, 0xab, 0xbf, 0x48, 0x08, 0x57, 0x38, 0xcb, 0xe9, 0xb9, 0xa9, 0xca, 0x09, 0x3a, 0x6a, 0xdb,
};

// setup the wireless mode; infrastructure - connect to AP; adhoc - connect to another WiFi device
//#define WIRELESS_MODE_INFRA  1
//#define WIRELESS_MODE_ADHOC 2
unsigned char wireless_mode = WIRELESS_MODE_INFRA;
// End of wireless configuration parameters ----------------------------------------
/*
 * uint16_t localPort = 1026;      // local port to listen on * fails SA: aborted
 * uint16_t dest_Port = 6000;      // local port to listen on * fails SA: aborted
 * Wireshark: 143  18.779493 10.0.1.57 10.0.1.64 TCP 58  1026 → 6000 [SYN] Seq=0 Win=429 Len=0 MSS=429
 * Wireshark: 144  18.779572 10.0.1.64 10.0.1.57 TCP 54  6000 → 1026 [RST, ACK] Seq=1 Ack=1 Win=0 Len=0
 * Wireshark: 465  33.282291 10.0.1.64 10.0.1.57 UDP 47  6000 → 1026  Len=5

 * uint16_t dest_Port = 80;      // local port to listen on * passes SA: connected
 *  === Initializing WiShield as Client to display DHCP Query on Serial Monitor ===
 *  Local port:1026
 *  Destination port:80
 *  Destination ip:10,0,1,64
 *  
 * WiFi Init OK; Start the DHCP query...
 * DHCP IP     : 10.0.1.57
 * DHCP GATEWAY: 10.0.1.1
 * DHCP NETMASK: 255.255.255.0
 * DHCP DNS    : 10.0.1.1
 * SA: connected / send
 * SA: newdata
 * SA: acked
 * SA: closed / timedout
 * 
 * Wireshark: 20  8.294431  Microchi_0a:a0:c7 Broadcast LLC 20  S P, func=RNR, N(R)=64; DSAP NULL LSAP Individual, SSAP NULL LSAP Command
 * Wireshark: 29  9.830348  Microchi_0a:a0:c7 Broadcast ARP 42  Who has 10.0.1.64? Tell 10.0.1.57
 * Wireshark: 30  9.830402  Apple_00:f9:7e  Microchi_0a:a0:c7 ARP 42  10.0.1.64 is at 00:26:bb:00:f9:7e
 * 
 * Wireshark: 38  11.791007 10.0.1.57 10.0.1.64 TCP 58  1026 → 80 [SYN] Seq=0 Win=429 Len=0 MSS=429
 * Wireshark: 39  11.791154 10.0.1.64 10.0.1.57 TCP 58  80 → 1026 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=1460
 * Wireshark: 41  11.798863 10.0.1.57 10.0.1.64 TCP 67  [TCP segment of a reassembled PDU]
 * Wireshark: 42  11.798946 10.0.1.64 10.0.1.57 TCP 54  80 → 1026 [ACK] Seq=1 Ack=14 Win=65535 Len=0
 * Wireshark: 43  11.804133 10.0.1.57 10.0.1.64 TCP 54  1026 → 80 [FIN, ACK] Seq=14 Ack=1 Win=429 Len=0
 * Wireshark: 44  11.804246 10.0.1.64 10.0.1.57 TCP 54  80 → 1026 [ACK] Seq=1 Ack=15 Win=65535 Len=0
 * Wireshark: 45  11.805637 10.0.1.64 10.0.1.57 TCP 54  80 → 1026 [FIN, ACK] Seq=1 Ack=15 Win=65535 Len=0
 * Wireshark: 46  11.810786 10.0.1.57 10.0.1.64 TCP 54  1026 → 80 [ACK] Seq=15 Ack=2 Win=429 Len=0
 * 
 * Wireshark: 51  11.950227 10.0.1.64 10.0.1.57 UDP 47  6000 → 1026  Len=5 
 *   (UDP message from Processing sketch port 6000 that was broadcasting on Mac ip 10.0.1.64)
 */
 
char  packetBuffer[255]; //buffer to hold incoming packet
char  ReplyBuffer[] = "acknowledged";       // a string to send back

// global data
boolean connectAndSendTCP = false;
uip_ipaddr_t srvaddr;

void setup()
{
  // Enable Serial output
  Serial.begin(115200);

  Serial.println ("\n=== Initializing WiShield as Client to display DHCP Query on Serial Monitor ===");

  Serial.print("Local port:");
  Serial.print(localPort);
  Serial.print("\nDestination port:");
  Serial.print(dest_Port);
  Serial.print("\nDestination ip:");
  Serial.print(dest_ip[0]); Serial.print(",");
  Serial.print (dest_ip[1]); Serial.print(",");
  Serial.print (dest_ip[2]); Serial.print(",");
  Serial.println (dest_ip[3]);
  Serial.println();

  WiFi.init();

  Serial.println("\nWiFi Init OK; Start the DHCP query...");
  uip_dhcp_request();
}

void loop()
{
  if (true == connectAndSendTCP) {
    connectAndSendTCP = false;
    // Address of server to connect to
    //uip_ipaddr(&srvaddr, 10,0,1,64);
    uip_ipaddr(&srvaddr, (dest_ip[0]), (dest_ip[1]), (dest_ip[2]), (dest_ip[3]));
    uip_connect(&srvaddr, HTONS(dest_Port));
  }

  WiFi.run();
}

extern "C" {
  // Process UDP UIP_APPCALL events
  void udpapp_appcall(void)
  {
    uip_dhcp_run();
  }

  // DHCP query complete callback
  void uip_dhcp_callback(const struct dhcp_state *s)
  {
    if (NULL != s) {
      // Set the received IP addr data into the uIP stack
      uip_sethostaddr(s->ipaddr); // DHCP assigned Arduino Wifi ip address
      uip_setdraddr(s->default_router); // code assigned gateway_ip[]
      uip_setnetmask(s->netmask); // code assigned subnet_mask[]
      // code assigned dnsaddr -> dns_ip[]
      // Print the received data - its quick and dirty but informative
      Serial.print("DHCP IP     : ");
      Serial.print(uip_ipaddr1(s->ipaddr), DEC);
      Serial.print(".");
      Serial.print(uip_ipaddr2(s->ipaddr), DEC);
      Serial.print(".");
      Serial.print(uip_ipaddr3(s->ipaddr), DEC);
      Serial.print(".");
      Serial.println(uip_ipaddr4(s->ipaddr), DEC);

      Serial.print("DHCP GATEWAY: ");
      Serial.print(uip_ipaddr1(s->default_router), DEC);
      Serial.print(".");
      Serial.print(uip_ipaddr2(s->default_router), DEC);
      Serial.print(".");
      Serial.print(uip_ipaddr3(s->default_router), DEC);
      Serial.print(".");
      Serial.println(uip_ipaddr4(s->default_router), DEC);

      Serial.print("DHCP NETMASK: ");
      Serial.print(uip_ipaddr1(s->netmask), DEC);
      Serial.print(".");
      Serial.print(uip_ipaddr2(s->netmask), DEC);
      Serial.print(".");
      Serial.print(uip_ipaddr3(s->netmask), DEC);
      Serial.print(".");
      Serial.println(uip_ipaddr4(s->netmask), DEC);

      Serial.print("DHCP DNS    : ");
      Serial.print(uip_ipaddr1(s->dnsaddr), DEC);
      Serial.print(".");
      Serial.print(uip_ipaddr2(s->dnsaddr), DEC);
      Serial.print(".");
      Serial.print(uip_ipaddr3(s->dnsaddr), DEC);
      Serial.print(".");
      Serial.println(uip_ipaddr4(s->dnsaddr), DEC);
    }
    else {
      Serial.println("DHCP NULL FALLBACK");
    }

    // Shut down DHCP
    uip_dhcp_shutdown();

    connectAndSendTCP = true;
  }


  char packet[] = "SocketAppDHCP";

  void socket_app_appcall(void)
  {
    if (uip_closed() || uip_timedout()) {
      Serial.println("SA: closed / timedout");
      uip_close();
      return;
    }
    if (uip_poll()) {
      Serial.println("SA: poll");
    }
    if (uip_aborted()) {
      Serial.println("SA: aborted");
    }
    if (uip_connected()) {
      Serial.println("SA: connected / send");
      uip_send(packet, strlen(packet));
    }
    if (uip_acked()) {
      Serial.println("SA: acked");
      uip_close();
    }
    if (uip_newdata()) {
      Serial.println("SA: newdata");
    }
    if (uip_rexmit()) {
      Serial.println("SA: rexmit");
      uip_send(packet, strlen(packet));
    }
  }

  // These uIP callbacks are unused for the purposes of this simple DHCP example
  // but they must exist.
  void socket_app_init(void)
  {
  }

  void udpapp_init(void)
  {
  }

  void dummy_app_appcall(void)
  {
  }
}

/*

# -- Beginning of python server script

import socket

HOST = ''                 # Symbolic name meaning all available interfaces
PORT = 3333               # Arbitrary non-privileged port
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((HOST, PORT))
s.listen(1)

try:
 while 1:
  conn, addr = s.accept()
  print 'Connected by', addr
  data = conn.recv(1024)
  if not data:
   continue
  print data
  conn.close()
except:
 conn.close()

# -- End of python script

*/
(Apr 28, 2016)

Tuesday, April 26, 2016

YellowJacket Arduino with WiShield Wifi (Part 20)

Investigating SocketApp with DNS —
The Examples:SocketAppDNS.ino sketch was modified to run on the local network. The resulting SocketAppDNS_V14_01.ino sketch compiles and runs on the YellowJacket Arduino using wifi to communicate with the local network and to access the remote LAX weather server to recover the “DNS addr received: 23.67.242.48”. The apps-conf.h file must be edited to use APP_SOCKAPP, APP_UDPAPP and UIP_DNS. The output from the app is shown near the top of the following sketch code:
/*
 * SocketAppDNS_V14_01
 * 2016-04-22
 * 84Park
 *
 * A simple socket application / DNS example for the WiShield
 * 
 *  Compiled with WiShield_V14 library (ref: github.com/hamityanik/WiShield_user_contrib) and Arduino 1.6.5
 *  This code works 2016-04-22 with YJ57 board with/without SD Card Shield
 *  Pre-calculated security_data[] inits in 6 seconds
 *
 * Example Output to Serial Monitor
 * for destURL[] = {"w1.weather.gov"};
 * 
=== Initializing WiShield as Client to display DNS Address on Serial Monitor ===

WiFi Init OK; Start the DNS query...
Servicing DNS query: udpapp_appcall() -> uip_dns_run()
Servicing DNS query: udpapp_appcall() -> uip_dns_run()
Servicing DNS query: udpapp_appcall() -> uip_dns_newdata()
DNS addr received: 23.67.242.42

 */
 
// Requires APP_SOCKAPP, APP_UDPAPP and UIP_DNS to be defined in apps-conf.h
//  APP_SOCKAPP  - for the TCP sockets components of the sketch
//  APP_UDPAPP   - for the UDP/DNS components of the sketch
//  UIP_DNS      - for the DNS components of the sketch

#include <WiShield.h>
extern "C" {
   #include "uip.h"
}

// Wireless configuration parameters ----------------------------------------
unsigned char local_ip[]    = {10, 0, 1, 57};   // IP address of WiShield
unsigned char gateway_ip[]  = {10, 0, 1, 1};   // router or gateway IP address
unsigned char subnet_mask[] = {255, 255, 255, 0}; // subnet mask for the local network
u16_t dns_ip[]              = {10, 0, 1, 1};   // DNS server addr
char ssid[]                 = {"Network"};   // max 32 bytes
unsigned char security_type = 5;  // 0 - open; 1 - WEP; 2 - WPA; 3 - WPA2; 4 - WPA Precalc; 5 - WPA2 Precalc

// Depending on your security_type, uncomment the appropriate type of security_data
// 0 - None (open)
// const char security_data[] PROGMEM = {};

// 1 - WEP
// UIP_WEP_KEY_LEN. 5 bytes for 64-bit key, 13 bytes for 128-bit key
// Only supply the appropriate key, do not specify 4 keys and then try to specify which to use
//const char security_data[] PROGMEM = { 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, };

// 2, 3 - WPA/WPA2 Passphrase
// 8 to 63 characters which will be used to generate the 32 byte calculated key
// Expect the g2100 to take 30 seconds to calculate the key from a passphrase
// const char security_data[] PROGMEM = {"Network_Passcode"};

// 4, 5 - WPA/WPA2 Precalc
// The 32 byte precalculate WPA/WPA2 key. This can be calculated in advance to save boot time
//const prog_char security_data[] PROGMEM = {
//    0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f,
//    0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1f, 0x1f,
//};
// Calc the Precalc:
// http://jorisvr.nl/wpapsk.html
// The derived key will appear in the form as a sequence of 64 hexadecimal digits.
/*0599967ce21000026c2cf439ea561c3d37abbf48085738cbe9b9a9ca093a6adb*/
const char security_data[] PROGMEM = {
  0x05, 0x99, 0x96, 0x7c, 0xe2, 0x10, 0x00, 0x02, 0x6c, 0x2c, 0xf4, 0x39, 0xea, 0x56, 0x1c, 0x3d,
  0x37, 0xab, 0xbf, 0x48, 0x08, 0x57, 0x38, 0xcb, 0xe9, 0xb9, 0xa9, 0xca, 0x09, 0x3a, 0x6a, 0xdb,
};

// setup the wireless mode; infrastructure - connect to AP; adhoc - connect to another WiFi device
//#define WIRELESS_MODE_INFRA  1
//#define WIRELESS_MODE_ADHOC 2
unsigned char wireless_mode = WIRELESS_MODE_INFRA;
// End of wireless configuration parameters ----------------------------------------

// The resolved DNS address
uint8 dnsAddr[] = {0,0,0,0};
char destURL[]  = {"w1.weather.gov"};

void setup()
{
   // Enable Serial output
   Serial.begin(115200);

  Serial.println ("\n=== Initializing WiShield as Client to display DNS Address on Serial Monitor ===");

   WiFi.init();
   
   Serial.println("\nWiFi Init OK; Start the DNS query...");
   uip_dns_conf(dns_ip);
   uip_dns_query(destURL);
}

void loop()
{
   WiFi.run();
}

extern "C" {
   // Process UDP UIP_APPCALL events
   void udpapp_appcall(void)
   {
      if(uip_udp_conn->rport == HTONS(53)) {
         if(uip_poll()) {
            Serial.println("Servicing DNS query: udpapp_appcall() -> uip_dns_run()");
            uip_dns_run();
         }
         if(uip_newdata()) {
            Serial.println("Servicing DNS query: udpapp_appcall() -> uip_dns_newdata()");
            uip_dns_newdata();
         }
      }
   }

   // DNS resolver will call this function for either succesful or failed DNS lookup
   // uip_dns_query() call (above) starts the chain of events leading to this callback
   void uip_dns_callback(char *name, u16_t *ipaddr)
   {
      if(NULL != ipaddr) {
         // TODO: probably a better way to do this...
         dnsAddr[0] = uip_ipaddr1(ipaddr);
         dnsAddr[1] = uip_ipaddr2(ipaddr);
         dnsAddr[2] = uip_ipaddr3(ipaddr);
         dnsAddr[3] = uip_ipaddr4(ipaddr);
         Serial.print("DNS addr received: "); 
         Serial.print(dnsAddr[0], DEC);
         Serial.print(".");
         Serial.print(dnsAddr[1], DEC);
         Serial.print(".");
         Serial.print(dnsAddr[2], DEC);
         Serial.print(".");
         Serial.println(dnsAddr[3], DEC);
      }
      else {
         Serial.println("DNS query failed");
      }
      
       // Shutdown DNS
       uip_dns_shutdown();
   }

   // These uIP callbacks are unused for the purposes of this simple DNS example
   // but they must exist.   
   void socket_app_init(void)
   {
   }

   void socket_app_appcall(void)
   {
   }

   void udpapp_init(void)
   {
   }

   void dummy_app_appcall(void)
   {
   }
}

(Apr 26, 2016)

Monday, April 25, 2016

YellowJacket Arduino with WiShield Wifi (Part 19)

Using WiShield DNS in YellowJacket Arduino Apps — 
What is DNS?
Ref: http://www.networksolutions.com/ explains: “Domain Name Servers (DNS) are the Internet’s equivalent of a phone book. They maintain a directory of domain names and translate them to Internet Protocol (IP) addresses.
This is necessary because, although domain names are easy for people to remember, computers or machines, access websites based on IP addresses.
Information from all the domain name servers across the Internet are gathered together and housed at the Central Registry. Host companies and Internet Service Providers interact with the Central Registry on a regular schedule to get updated DNS information.
When you type in a web address, e.g., www.somePlace.com, your Internet Service Provider views the DNS associated with the domain name, translates it into a machine friendly IP address (for example 216.168.224.234 is the IP for somePlace.com) and directs your Internet connection to the correct website.
After you register a new domain name or when you update the DNS servers on your domain name, it usually takes about 12-36 hours for the domain name servers world-wide to be updated and able to access the information. This 36-hour period is referred to as propagation.”
As discussed in Part 18 of this series, WiShield Version 1.4 adds DNS and pre-calculated security key functions. The first two examples, will show the differences of sketch code using WPA security and pre-calculated security keys. The sketch was downloaded using the USB/FTDI programming cable to the YellowJacket (YJ57) which communicated with the local network over wifi to request LAX weather data from the remote server. The ip for LAX weather was set to 0.0.0.0 and DNS used the “w1.weather.gov” to identify the ip address which was returned as 65.202.58.9 or 65.202.58.54.
The following example, WiServerDNS_V14_01.ino, shows the complete sketch which uses the original WPA/WEP security. Be sure to edit apps-conf.h to use the necessary APP_WISERVER, APP_UDPAPP, and UIP_DNS code.
As discussed in Part 18 of this series, WiShield Version 1.4 adds DNS and pre-calculated security key functions. The first two examples, will show the differences of sketch code using WPA security and pre-calculated security keys. The sketch was downloaded using the USB/FTDI programming cable to the YellowJacket (YJ57) which communicated with the local network over wifi to request LAX weather data from the remote server.
The following example, WiServerDNS_V14_01.ino, shows the complete sketch which uses the original WPA/WEP security. Be sure to edit apps-conf.h to use the necessary APP_WISERVER, APP_UDPAPP, and UIP_DNS code.
/*
 * WiServerDNS_V14_01
 * 2016-04-21
 * 84Park
 * 
 * A simple sketch that uses WiServer to get the hourly weather data from LAX and prints
 * it via the Serial API
 * 
 *  Compiled with WiShield_V14 library (ref: github.com/hamityanik/WiShield_user_contrib) and Arduino 1.6.5
 *  This code works 2016-04-21 with YJ57 board with/without SD Card Shield
 *  Standard calculated security_data[] inits in 30 seconds
 *  
 */
 
// Requires APP_WISERVER, APP_UDPAPP and UIP_DNS to be defined in apps-conf.h
//  APP_WISERVER - for the WiServer components of the sketch
//  APP_UDPAPP   - for the UDP/DNS components of the sketch
//  UIP_DNS      - for the DNS components of the sketch

#include <WiServer.h>
extern "C" {
  #include "uip.h"
}
// uip.h adds u16_t typedef

// Wireless configuration parameters ----------------------------------------
unsigned char local_ip[]    = {10, 0, 1, 57};   // IP address of WiShield
unsigned char gateway_ip[]  = {10, 0, 1, 1};   // router or gateway IP address
unsigned char subnet_mask[] = {255,255,255,0}; // subnet mask for the local network
u16_t dns_ip[]              = {10, 0, 1, 1};   // DNS server addr
char ssid[]                 = {"Network"};   // max 32 bytes
unsigned char security_type = 3;               // 0 - open; 1 - WEP; 2 - WPA; 3 - WPA2

// 2, 3 - WPA/WPA2 Passphrase
// 8 to 63 characters which will be used to generate the 32 byte calculated key
// Expect the g2100 to take 30 seconds to calculate the key from a passphrase
const char security_data[] PROGMEM = {"Network_Passcode"};

// WPA/WPA2 passphrase
//const char security_passphrase[] PROGMEM = {"Network_Passcode"}; // max 64 characters

// WEP 128-bit keys
const uint8_t wep_keys[] PROGMEM = { 
 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, // Key 0
 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // Key 1
 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // Key 2
 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 // Key 3
};

// setup the wireless mode; infrastructure - connect to AP; adhoc - connect to another WiFi device
#define WIRELESS_MODE_INFRA 1
#define WIRELESS_MODE_ADHOC 2
unsigned char wireless_mode = WIRELESS_MODE_INFRA;
unsigned char ssid_len;
unsigned char security_passphrase_len;
// End of wireless configuration parameters ----------------------------------------


// Flag to know when the DNS query has completed
boolean dnsCalledBack = false;

// Function that prints data from the server
void printData(char* data, int len) {
  
  // Print the data returned by the server
  // Note that the data is not null-terminated, may be broken up into smaller packets, and 
  // includes the HTTP header. 
  while (len-- > 0) {
    Serial.print(*(data++));
  }
}


// Hardcoded IP Address for www.weather.gov
// If DNS lookup is succesful this address will be set in uip_dns_callback
// Otherwise use hardcoded working value...
// uint8 ip[] = {140,90,113,200};
// uint8 ip[] = {71,231,196,153};
uint8 ip[] = {23, 67, 242, 42}; // IP Address for w1.weather.gov (2016-02-25)
// Returned:
//    DNS ADDR RECEIVED: 65.202.58.9
//    DNS ADDR RECEIVED: 65.202.58.54

char destURL[]                 = {"w1.weather.gov"};
char destGet[]                 = {"/data/METAR/KLAX.1.txt"};

// A request that gets the latest METAR weather data for LAX
//GETrequest getWeather(ip, 80, "w1.weather.gov", "/data/METAR/KLAX.1.txt");
//GETrequest getWeather(ip, 80, destURL, "/data/METAR/KLAX.1.txt");
GETrequest getWeather(ip, 80, destURL, destGet);

void setup() {

  // Enable Serial output
  Serial.begin(115200);

    Serial.println (" ");
    Serial.println ("=== Initializing WiServer as Client to display WX web page on Serial Monitor ===");

  // Initialize WiServer (we'll pass NULL for the page serving function since we don't need to serve web pages)
  WiServer.init(NULL);

  // Ask WiServer to generate log messages (optional)
  //WiServer.enableVerboseMode(true);

  // Have the processData function called when data is returned by the server
  getWeather.setReturnFunc(printData);

    Serial.println (" ");
    Serial.println ("=== WiServer is running... ===");
    Serial.println (">>> Request response on 10 sec intervals... ");
    Serial.println (" ");


  // Start the DNS query
  uip_dns_conf(dns_ip);
  //uip_dns_query("www.weather.gov");
  uip_dns_query("w1.weather.gov");
  //uip_dns_query("slacklab.org");
}

// Time (in millis) when the data should be retrieved 
long updateTime = 0;

void loop()
{
  // Check if it's time to get an update
  if (true == dnsCalledBack && millis() >= updateTime) {
    // Shutdown DNS
    uip_dns_shutdown();
    // Call WiServer to fetch a page
    getWeather.submit();    
    // Get another update one hour from now
    updateTime += 1000 * 60 * 60;
  }
  
  // Run WiServer
  WiServer.server_task();
 
  delay(10);
}

extern "C" {

   // Process UDP UIP_APPCALL events
   void udpapp_appcall(void)
   {
      if(uip_poll()) {
         uip_dns_run();
      }
      if(uip_newdata()) {
         uip_dns_newdata();
      }
   }

   // DNS resolver will call this function for either succesful or failed DNS lookup
   // uip_dns_query() call (above) starts the chain of events leading to this callback
   void uip_dns_callback(char *name, u16_t *ipaddr)
   {
      dnsCalledBack = true;
      
      if(NULL != ipaddr) {
         // TODO: probably a better way to do this...
         ip[0] = uip_ipaddr1(ipaddr);
         ip[1] = uip_ipaddr2(ipaddr);
         ip[2] = uip_ipaddr3(ipaddr);
         ip[3] = uip_ipaddr4(ipaddr);
         Serial.print("DNS ADDR RECEIVED: "); 
         Serial.print(ip[0], DEC);
         Serial.print(".");
         Serial.print(ip[1], DEC);
         Serial.print(".");
         Serial.print(ip[2], DEC);
         Serial.print(".");
         Serial.println(ip[3], DEC);
      }
      else {
         Serial.println("DNS NULL - FALLBACK TO DEFAULT IP ADDRESS");
      }
   }

   // Not needed for this example but must exist
   void udpapp_init(void)
   {
   }

   // Not needed for this example but must exist
   void dummy_app_appcall(void)
   {
   }
}
The next example , WiServerDNS_V14_03.ino, shows the same sketch modified to use pre-calculated security keys. As can be seen, only the security at the top of the sketch changes and the latter code remains the same. So depending on whether your choice is one type of security or another can be made with some simple cut and paste editing. This example sketch, also shows the output to the Serial Monitor, near the top of the sketch code. The code is large and only 134 bytes of dynamic memory remains on an Arduino 328P device.
/*
 * WiServerDNS_V14_03
 * 2016-04-21
 * 84Park
 *
 * A simple sketch that uses WiServer to get the hourly weather data from LAX and prints
 * it via the Serial API
 *
 * WiServerDNS sketch from V14 Examples sketches
 * Differences from previous V13 sketch code.
 *   function getWeather.setuIP(ipaddr); in void uip_dns_callback(char *name, u16_t *ipaddr)
 *   prints out the ipaddr that is returned.
 *
 *  Compiled with WiShield_V14 library (ref: github.com/hamityanik/WiShield_user_contrib) and Arduino 1.6.5
 *  This code works 2016-04-21 with YJ57 board with/without SD Card Shield
 *  Pre-calculated security_data[] inits in 6 seconds
 *
 * Example Output to Serial Monitor
 * 
=== Initializing WiServer as Client to display WX web page on Serial Monitor ===
WiServer init called

=== WiServer is running... ===
>>> Request response on 10 sec intervals...

Servicing DNS query: udpapp_appcall() -> uip_dns_run()
Servicing DNS query: udpapp_appcall() -> uip_dns_run()
Servicing DNS query: udpapp_appcall() -> uip_dns_newdata()
DNS ADDR RECEIVED: 65.202.58.54
Got connection for w1.weather.gov
Connected to w1.weather.gov
TX 87 bytes
0 - 86 of 87
RX 0 bytes from w1.weather.gov
RX 387 bytes from w1.weather.gov
HTTP/1.0 200 OK
Server: Apache/2.2.15 (Red Hat)
X-NIDS-ServerID: www11.md
Last-Modified: Fri, 22 Apr 2016 09:57:01 GMT
Accept-Ranges: bytes
Content-Length: 117
Content-Type: text/plain; charset=iso-8859-1
Date: Fri, 22 Apr 2016 10:21:14 GMT
Connection: close

000
SAUS80 KWBC 221000 RRA
MTRLAX
METAR KLAX 220953Z 00000KT 10SM FEW015 SCT250 16/13 A2990 RMK AO2 SLP122
T01610128
Ended connection with w1.weather.gov
 *
 */

// Requires APP_WISERVER, APP_UDPAPP and UIP_DNS to be defined in apps-conf.h
//  APP_WISERVER - for the WiServer components of the sketch
//  APP_UDPAPP   - for the UDP/DNS components of the sketch
//  UIP_DNS      - for the DNS components of the sketch

#include <WiServer.h>
extern "C" {
#include "uip.h"
}
// uip.h adds u16_t typedef

// Wireless configuration parameters ----------------------------------------
unsigned char local_ip[]    = {10, 0, 1, 57};   // IP address of WiShield
unsigned char gateway_ip[]  = {10, 0, 1, 1};   // router or gateway IP address
unsigned char subnet_mask[] = {255, 255, 255, 0}; // subnet mask for the local network
u16_t dns_ip[]              = {10, 0, 1, 1};   // DNS server addr
char ssid[]                 = {"Network"};   // max 32 bytes
unsigned char security_type = 5;  // 0 - open; 1 - WEP; 2 - WPA; 3 - WPA2; 4 - WPA Precalc; 5 - WPA2 Precalc

// Depending on your security_type, uncomment the appropriate type of security_data
// 0 - None (open)
// const char security_data[] PROGMEM = {};

// 1 - WEP
// UIP_WEP_KEY_LEN. 5 bytes for 64-bit key, 13 bytes for 128-bit key
// Only supply the appropriate key, do not specify 4 keys and then try to specify which to use
//const char security_data[] PROGMEM = { 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, };

// 2, 3 - WPA/WPA2 Passphrase
// 8 to 63 characters which will be used to generate the 32 byte calculated key
// Expect the g2100 to take 30 seconds to calculate the key from a passphrase
// const char security_data[] PROGMEM = {"Network_Passcode"};

// 4, 5 - WPA/WPA2 Precalc
// The 32 byte precalculate WPA/WPA2 key. This can be calculated in advance to save boot time
//const prog_char security_data[] PROGMEM = {
//    0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f,
//    0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1f, 0x1f,
//};
// Calc the Precalc:
// http://jorisvr.nl/wpapsk.html
// The derived key will appear in the form as a sequence of 64 hexadecimal digits.
/*0599967ce21000026c2cf439ea561c3d37abbf48085738cbe9b9a9ca093a6adb*/
const char security_data[] PROGMEM = {
  0x05, 0x99, 0x96, 0x7c, 0xe2, 0x10, 0x00, 0x02, 0x6c, 0x2c, 0xf4, 0x39, 0xea, 0x56, 0x1c, 0x3d,
  0x37, 0xab, 0xbf, 0x48, 0x08, 0x57, 0x38, 0xcb, 0xe9, 0xb9, 0xa9, 0xca, 0x09, 0x3a, 0x6a, 0xdb,
};

// setup the wireless mode; infrastructure - connect to AP; adhoc - connect to another WiFi device
//#define WIRELESS_MODE_INFRA  1
//#define WIRELESS_MODE_ADHOC 2
unsigned char wireless_mode = WIRELESS_MODE_INFRA;
// End of wireless configuration parameters ----------------------------------------

// Flag to know when the DNS query has completed
boolean dnsCalledBack = false;

// Function that prints data from the server
void printData(char* data, int len) {

  // Print the data returned by the server
  // Note that the data is not null-terminated, may be broken up into smaller packets, and
  // includes the HTTP header.
  while (len-- > 0) {
    Serial.print(*(data++));
  }
}

// Hardcoded IP Address for www.weather.gov
// If DNS lookup is succesful this address will be set in uip_dns_callback
// Otherwise use hardcoded working value...
// uint8 ip[] = {23, 67, 242, 42}; // IP Address for w1.weather.gov (2016-02-25)
uint8 ip[] = {0, 0, 0, 0}; // IP Address 0.0.0.0
// Returned:
//    DNS ADDR RECEIVED: 65.202.58.9
//    DNS ADDR RECEIVED: 65.202.58.54

char destURL[]                 = {"w1.weather.gov"};
char destGet[]                 = {"/data/METAR/KLAX.1.txt"};

// A request that gets the latest METAR weather data for LAX
//GETrequest getWeather(ip, 80, "w1.weather.gov", "/data/METAR/KLAX.1.txt");
//GETrequest getWeather(ip, 80, destURL, "/data/METAR/KLAX.1.txt");
GETrequest getWeather(ip, 80, destURL, destGet);

void setup() {

  // Enable Serial output
  Serial.begin(115200);

  Serial.println ("\n=== Initializing WiServer as Client to display WX web page on Serial Monitor ===");

  // Initialize WiServer (we'll pass NULL for the page serving function since we don't need to serve web pages)
  WiServer.init(NULL);

  // Ask WiServer to generate log messages (optional)
  //WiServer.enableVerboseMode(true);
  // not available in V1.4

  // Have the processData function called when data is returned by the server
  getWeather.setReturnFunc(printData);

  Serial.println ("\n=== WiServer is running... ===");
  Serial.println (">>> Request response on 10 sec intervals... \n");

  // Start the DNS query
  uip_dns_conf(dns_ip);
  //uip_dns_query("www.weather.gov");
  //uip_dns_query("slacklab.org");
  //uip_dns_query("w1.weather.gov");
  uip_dns_query(destURL);
}

// Time (in millis) when the data should be retrieved
long updateTime = 0;

void loop()
{
  // Check if it's time to get an update
  if (true == dnsCalledBack && millis() >= updateTime) {
    // Shutdown DNS
    uip_dns_shutdown();
    // Call WiServer to fetch a page
    getWeather.submit();
    // Get another update one hour from now
    updateTime += 1000 * 60 * 60; // 10 sec ; // 60 * 60; // hour
  }

  // Run WiServer
  WiServer.server_task();

  delay(10);
}

extern "C" {

  // Process UDP UIP_APPCALL events
  void udpapp_appcall(void)
  {
    if (uip_poll()) {
      Serial.println("Servicing DNS query: udpapp_appcall() -> uip_dns_run()");
      uip_dns_run();
    }
    if (uip_newdata()) {
      Serial.println("Servicing DNS query: udpapp_appcall() -> uip_dns_newdata()");
      uip_dns_newdata();
    }
  }

  // DNS resolver will call this function for either succesful or failed DNS lookup
  // uip_dns_query() call (above) starts the chain of events leading to this callback
  void uip_dns_callback(char *name, u16_t *ipaddr)
  {
    dnsCalledBack = true;

    if (NULL != ipaddr) {
      // better way: set the fetched ip
      getWeather.setuIP(ipaddr);

      /*
       // TODO: probably a better way to do this...
       ip[0] = uip_ipaddr1(ipaddr);
       ip[1] = uip_ipaddr2(ipaddr);
       ip[2] = uip_ipaddr3(ipaddr);
       ip[3] = uip_ipaddr4(ipaddr);
       Serial.print("DNS ADDR RECEIVED: ");
       Serial.print(ip[0], DEC);
       Serial.print(".");
       Serial.print(ip[1], DEC);
       Serial.print(".");
       Serial.print(ip[2], DEC);
       Serial.print(".");
       Serial.println(ip[3], DEC);
      */
    }
    else {
      Serial.println("DNS NULL - FALLBACK TO DEFAULT IP ADDRESS");
    }
  }

  // Not needed for this example but must exist
  void udpapp_init(void)
  {
  }

  // Not needed for this example but must exist
  void dummy_app_appcall(void)
  {
  }
}

(Apr 25, 2016)

Friday, April 22, 2016

YellowJacket Arduino with WiShield Wifi (Part 18)

Improvements of the WiShield Library — 
The WiShield Library (GregEigsti et al., WiShield_user_contrib) has many versions on GitHub since the original Library was contributed by AsyncLabs (2009). It takes some effort to sort through the various user contributions to determine if the code changes affect what a developer needs for their app. Some improvements to the WiShield Library that seem useful include: adding DHCP/DNS rather than having to use fixed ip addresses, UDP was reported to have been improved, and, a 32-byte precalculated security key reduces Wifi init time from about 30 seconds to about 6 seconds. DEBUGU was added which implemented printx/printlnx functions for sending char strings to the sketch for printing debug information on Serial Monitor. In addition, the 6-7 digit limit of accuracy for the dtostrf function was added to include a function of using long integer values for representing float numbers.
Adding DHCP/DNS
Ref: https://github.com/asynclabs/WiShield_user_contrib (Named Version 1.2 for clarity)
WiShield V1.2 was downloaded and tested to determine if UDP had been improved over WiShield V3 (see Part 4 of this series for a discussion about Version 3). V1.2 includes DNS and additional UDP code. V1.2 required code fixes typical of pre-Arduino_V1.6.5. Once those fixes were made, library Example sketches WiServerDNS_V12_01 and WiFiScan_V12_01 were compiled and run on YJ57 connected through the USB FTDI programmer port.
The WiServerDNS_V12_01 sketch successfully used the following code which used DNS in the GETrequest to request the current LAX weather from w1.weather.gov:
/*****************************************************************************/
// Using DNS to return weather data
uint8 ip[] = {23, 67, 242, 42}; // IP Address for w1.weather.gov (2016-02-25)
// Returned:
//    DNS ADDR RECEIVED: 65.202.58.9
//    DNS ADDR RECEIVED: 65.202.58.54

char destURL[]                 = {"w1.weather.gov"};
char destGet[]                 = {"/data/METAR/KLAX.1.txt"};

// A request that gets the latest METAR weather data for LAX
//GETrequest getWeather(ip, 80, "w1.weather.gov", "/data/METAR/KLAX.1.txt");
GETrequest getWeather(ip, 80, destURL, destGet);
// ...
/*****************************************************************************/
WiFiScan_V12_01 sketch returned the rssi: values for ssid:’s that were identified as the app ran.
The following Data Logger sketches also run successfully with WiShield V1.2 installed:
  • ds1307_RTC – real time clock app
  • ADX_dumpFram_01 – FRAM memory report
  • SPI_SD_FRAM_Wifi_test – Checks SPI switching with several devices connected
  • RD_RamDiskToSdFat – File TEST.CSV was written to FRAM memory
  • ADX_dumpFram_01 was rerun to view FRAM memory after TEST.CSV write
WiShield V1.3 with DNS
Ref: github.com/CapnBry/WiShield_user_contrib (Named Version 1.3 for clarity)
WiShield V1.3 was downloaded and tested to determine if UDP had been improved over WiShield V3. V1.3 includes DNS and additional UDP code. V1.3 required code fixes typical of pre-Arduino_V1.6.5. Once those fixes were made, sketches WiServerDNS_V13_01 and WiFiScan_V13_01 were compiled and run on YJ57 connected through the USB FTDI programmer port.
V1.3 has many changes to V1.2 that need to be assessed if they make any value added improvements.
A 32-byte precalculated security key was generated and tested in WiServerDNS_V13_02 and WiFiScan_V13_02. The init time was significantly reduced.
“– Merges all the WEP / WPA into one variable, called security_data.
This saves on code space and the user can only use one at a time
anyway
— Adds WPA / WPA2 Precalc as a security type.
This is a 32 byte precalculated key so the g2100 doesn’t have to do it on
every boot. Reduces startup time from 35 seconds to 6 seconds.”
WiShield V1.4 with DNS, port to IDE 1.6.7
Ref: https://github.com/hamityanik/WiShield_user_contrib (2016-01-05, Named Version 1.4 for clarity)
The best part about this version is that it compiles without errors on Arduino Uno IDE 1.6.5. This version was added to the local Arduino/Libraries as WiShield_V14 and has become the current WiShield development library. In this V1.4, the previous requirement to edit uip-conf.h to set the “UIP_UDP_ENABLED 0/1” was moved to apps-cont.h and is set as part of selecting “#define APP_UDPAPP”. “boolean” definition needs to be commented out (//typedef uint8_t boolean;) in witypes.h to fix conflicts between BOOL and boolean. Also, other edits for JSON in previous parts of this series were made:
In strings.h, add:
“const char PROGMEM contentTypeJson[] = {“Content-Type: application/json; charset=utf-8″};”
, and in WiServerIO.h/.cpp
“extern const char PROGMEM contentTypeJson[];”
“int fType = 0;”
“void setContentType(int value){ fType = value; }”
also need to be made.
Versions 1.3 and 1.4, can use the 32-byte precalculated security key, rather than previous WPA/WEP keys. The user must calculate the security key and modify Arduino sketches to use the new key. See Examples: SocketApp.ino for details about how to create the security key and how to use it in a sketch.
WiServerDNS_V12_01 sketch (portion above) was successfully compiled with the local network ip’s and 32-byte precalculated security key and run on YJ57 connected through the USB FTDI programmer port.
Other versions of WiShield_user_contrib which might be useful:
WiShield_Cameronr_user_contrib-master 2011-08-26
Ref: github.com/cameronr/WiShield_user_contrib
WiShield_SamCleaver_user_contrib-master 2012-10-30
Ref: github.com/SamCleaver/WiShield_user_contrib
Future parts to this series will contain code examples for using the Version 1.4 WiShield code. Feel free to download Version 1.4, generate a Security Key, and get some demo examples working. That will help find bugs in variable definitions and inconsistencies that might exist.

(Apr 22, 2016)

Friday, April 8, 2016

YellowJacket Arduino with WiShield Wifi (Part 17)

Adding Memory to the YellowJacket Arduino —
Early attempts to transmit sensor data from YJ56 over wifi to YJ57 and then write those data to the SD Card failed to compile due to not having enough dynamic memory on the Uno to include the SD Card library code. Web pages about Arduino dynamic memory clearly stated that the 328P processor could not have its dynamic memory expanded. My notes state:
“Arduino Memory: The Atmega 328 used in the Arduino UNO use a relatively pure Harvard architecture. Programs are stored in Flash memory and data is stored in SRAM. Flash memory is used to store the program image and any initialized data. You can execute program code from flash, but you can’t modify data in flash memory from your executing code. To modify the data, it must first be copied into SRAM. Flash memory is non-volatile, so your program will still be there when the system is powered off.”
One of the bigger offenders of using valuable dynamic (flash) memory is using print() and println() statements for Serial Monitor. A simple fix is to store the text strings that are to be printed in SRAM, e.g., (Serial.println(F(“&gt;&gt; Listening for connections…”));, where (F(“TEXT”) will be saved as data in SRAM rather than in flash.
One good reference that discusses Arduino memory is http://arduino.stackexchange.com/questions/221/what-can-i-do-if-i-run-out-of-flash-memory-or-sram.
A quick review of various memory technologies pointed to FRAM as one that had many benefits: 1) non-volatile; 2) fast; 3) easy read and write functions; and, 4) SPI hardware interface. The Adafruit SPI FRAM breakout board (Ref: https://learn.adafruit.com/adafruit-spi-fram-breakout/overview) was installed using a header and pins on an SD Card Shield. FRAM CS pin was wired to Shield pin D6. The breakout board FRAM chip, model MB85RS64V, is 8,192 words x 8 bits (SPI) with 16-bit addressing. Fujitsu currently manufactures larger FRAM chips (MB85RS2MT) with 262,144 words ×8 bits (SPI) with 24-bit addressing.
Adafruit_FRAM_SPI Library
To get started, the Adafruit_FRAM_SPI Library was downloaded to test the FRAM memory which worked with no problems. Each byte of FRAM memory is individually addressable from address:(uint16_t byteAddress(0-8191)). Command examples include:
  • fram.writeEnable(true/false);
  • fram.write(0xaddress, (uint8_t *)”Your text”, 10);
  • value=fram.read8(byteAddress);
Hackscribble_Ferro Library
Ref: https://github.com/hackscribble/hackscribble-ferro-library
Quoting the Ref: The Hackscribble_Ferro library makes it simple to use the FRAM in an Arduino sketch, for temporary storage while the sketch is running, for long-term storage which keeps its contents even if the Arduino is powered off, or a combination of these.
Hackscribble_Ferro provides functions for writing blocks of data to the FRAM, reading them back and controlling the FRAM. It assumes that your sketch will manage the details of how it uses the available FRAM memory.
Alternatively, Hackscribble_FerroArray lets you create arrays of data which are stored in the FRAM. For each array, you choose the type of data (e.g. byteintegerfloat, etc) and how many items are in the array (so long as you don’t exceed the maximum size of the FRAM).
Hackscribble_FerroArray takes care of how the arrays are stored in the FRAM, and you use simple functions to read and write your data.
Example sketch HF_test ran 14 read/write tests successfully on the FRAM MB85RS64V.
RamDisk Library
Ref: https://github.com/greiman/RamDisk
Ref: http://forum.arduino.cc/index.php?topic=229485.0
RamDisk provides a FAT style file system on external RAM devices. The RamDisk library was written for 24-bit FRAMs (MB85RS2MT) but the MB85RS64V uses 16-bit addresses. Quite a bit of coding was required to get RamDisk operational for all the test sketches. While the 8K MB85RS64V worked as a RamDisk, the MB85RS2MT would be a much better choice for a file storage system.
To use the MB85RS64V, initially each of the RamDisk library files was modified by changing unit32_t to uint16_t. In the end, only the modified MB85RS2MT.h, MB85RS2MT.cpp, and T_MB85RS2MT.h files were saved as an MB85RS64V library which would run the example sketches with the original RamDisk library. There may be lingering address issues in the MB85RS64V library which will have to be fixed.
Most issues in the Example sketches were linked to the limited memory of the 64 kbit MB85RS64V FRAM. There are lingering issues of device memory over-runs that are not trapped and reported by the code, both with the RamDisk Library and the modified SD Library provided with the RamDisk distribution. Sketch results using 16-bit MB85RS64V library and unmodified RamDisk library:
  • RD_HelloRamDisk OK
  • RD_testFram OK
  • RD_FramPerfTest OK
  • RD_RamTestPrint OK
  • RD_RamStdioBench OK
  • RD_RamByteReadWrite OK
  • RD_RamDiskLoggerTest_a OK
  • RD_RamDiskToSdFat OK
One handy sketch, ADX_dumpFram_01, was created to examine contents of the FRAM memory. The sketch uses one or both, Adafruit FRAM Library or Ramdisk Library, to read the FRAM addresses and print the contents as HEX, DEC, and ASCii. The beginning and end memory addresses can be set or the full memory can be displayed on Serial Monitor.
/*
 * 2016-04-06
 * ADX_dumpFram_01 for Arduino UNO, IDE 1.6.5
 *
 * If startRead > 0, startRead is adjusted to the closest buf read address.
 *
 * The ADX_dumpFram sketch reads an external FRAM on SPI
 * The code was developed using an Adafruit FRAM MB85RS64V breakout board
 *
 * The sketch works with the Adafruit MB85RS64V FRAM Library
 * or the RamDisk FRAM Library modified for the
 * MB85RS64V 16 bit Version
 *
 * Design:
 * Creates 8/64 byte buffer
 * Fills buffer with data from FRAM
 * Prints data, one byte at a time as HEX, DEC, and ASC
 *
 * This sketch reads/prints individual bytes of external memory
 *  As written, it does not process 2-byte or 4-byte numbers
 *  or more complex data structures
 *
 *  The sketch does not modify FRAM memory contents
 *
 * Sketch Example Output for MB85RS64 FRAM memory:
 * framTotalBytes = 8192;
 * startRead = 7938;
 * framEndByte = 8044;
 *
Type any character to begin

  ** Initialized Adafruit SPI FRAM **

 0x1F00: 15 16 17 18 19 1A 1B 1C  d7936:  21  22  23  24  25  26  27  28  a7936: . . . . . . . . 
 0x1F08: 1D 1E 1F 20 21 22 23 24  d7944:  29  30  31  32  33  34  35  36  a7944: . . . . ! " # $ 
 0x1F10: 25 26 27 28 29 2A 2B 2C  d7952:  37  38  39  40  41  42  43  44  a7952: % & ' ( ) * + , 
 0x1F18: 2D 2E 2F 30 31 32 33 34  d7960:  45  46  47  48  49  50  51  52  a7960: - . / 0 1 2 3 4 
 0x1F20: 35 36 37 38 39 3A 3B 3C  d7968:  53  54  55  56  57  58  59  60  a7968: 5 6 7 8 9 : ; < 
 0x1F28: 3D 3E 3F 40 41 42 43 44  d7976:  61  62  63  64  65  66  67  68  a7976: = > ? @ A B C D 
 0x1F30: 45 46 47 48 49 4A 4B 4C  d7984:  69  70  71  72  73  74  75  76  a7984: E F G H I J K L 
 0x1F38: 4D 4E 4F 50 51 52 53 54  d7992:  77  78  79  80  81  82  83  84  a7992: M N O P Q R S T 
 0x1F40: 55 56 57 58 59 5A 5B 5C  d8000:  85  86  87  88  89  90  91  92  a8000: U V W X Y Z [ \ 
 0x1F48: 5D 5E 5F 60 61 62 63 64  d8008:  93  94  95  96  97  98  99 100  a8008: ] ^ _ ` a b c d 
 0x1F50: 65 66 67 68 69 6A 6B 6C  d8016: 101 102 103 104 105 106 107 108  a8016: e f g h i j k l 
 0x1F58: 6D 6E 6F 70 71 72 73 74  d8024: 109 110 111 112 113 114 115 116  a8024: m n o p q r s t 
 0x1F60: 75 76 77 78 79 7A 7B 7C  d8032: 117 118 119 120 121 122 123 124  a8032: u v w x y z { . 
 0x1F68: 7D 7E 7F 80 81 82 83 84  d8040: 125 126 127 128 129 130 131 132  a8040: . . . . . . . . 

  ** Initialized RamDisk SPI FRAM **

 0x1F00: 15 16 17 18 19 1A 1B 1C  d7936:  21  22  23  24  25  26  27  28  a7936: . . . . . . . . 
 0x1F08: 1D 1E 1F 20 21 22 23 24  d7944:  29  30  31  32  33  34  35  36  a7944: . . . . ! " # $ 
 0x1F10: 25 26 27 28 29 2A 2B 2C  d7952:  37  38  39  40  41  42  43  44  a7952: % & ' ( ) * + , 
 0x1F18: 2D 2E 2F 30 31 32 33 34  d7960:  45  46  47  48  49  50  51  52  a7960: - . / 0 1 2 3 4 
 0x1F20: 35 36 37 38 39 3A 3B 3C  d7968:  53  54  55  56  57  58  59  60  a7968: 5 6 7 8 9 : ; < 
 0x1F28: 3D 3E 3F 40 41 42 43 44  d7976:  61  62  63  64  65  66  67  68  a7976: = > ? @ A B C D 
 0x1F30: 45 46 47 48 49 4A 4B 4C  d7984:  69  70  71  72  73  74  75  76  a7984: E F G H I J K L 
 0x1F38: 4D 4E 4F 50 51 52 53 54  d7992:  77  78  79  80  81  82  83  84  a7992: M N O P Q R S T 
 0x1F40: 55 56 57 58 59 5A 5B 5C  d8000:  85  86  87  88  89  90  91  92  a8000: U V W X Y Z [ \ 
 0x1F48: 5D 5E 5F 60 61 62 63 64  d8008:  93  94  95  96  97  98  99 100  a8008: ] ^ _ ` a b c d 
 0x1F50: 65 66 67 68 69 6A 6B 6C  d8016: 101 102 103 104 105 106 107 108  a8016: e f g h i j k l 
 0x1F58: 6D 6E 6F 70 71 72 73 74  d8024: 109 110 111 112 113 114 115 116  a8024: m n o p q r s t 
 0x1F60: 75 76 77 78 79 7A 7B 7C  d8032: 117 118 119 120 121 122 123 124  a8032: u v w x y z { . 
 0x1F68: 7D 7E 7F 80 81 82 83 84  d8040: 125 126 127 128 129 130 131 132  a8040: . . . . . . . . 
 0x1F70: 85 86 87 88 89 8A 8B 8C  d8048: 133 134 135 136 137 138 139 140  a8048: . . . . . . . . 
 0x1F78: 8D 8E 8F 90 91 92 93 94  d8056: 141 142 143 144 145 146 147 148  a8056: . . . . . . . . 
Done

 *
 */


#include <RamDisk.h> // ref: https://github.com/greiman/RamDisk
#include <MB85RS64V.h> // ref: https://github.com/greiman/RamDisk
#include "SPI.h" // Arduino IDE
#include "Adafruit_FRAM_SPI.h" // ref: Adafruit.com

uint8_t buf[64];                // used by RamDisk code
uint8_t aBuf[8];                // used by Adafruit code
uint32_t framTotalBytes = 8192; // device total FRAM MB85RS64V 8192 byte memory
uint32_t framEndByte = framTotalBytes;    // Stop location chosen for total memory report
//uint32_t framEndByte = 8044;    // Stop location chosen for subset report
uint32_t startByte = 0;       // Start location for examining memory
//uint32_t startByte = 7938;      // Start location for examining subset of memory
uint32_t bufNum = 0;            // number of the current buf array 8192/64=128 buf reads
uint32_t framMemPtr;            // FRAM address pointer [0...framTotalBytes]
uint8_t value;                  // byte value read from FRAM

const uint8_t FRAM_CS = 6;          // FRAM SPI CS pin

// Option: SPI FRAM using RamDisk library
T_MB85RS64V<FRAM_CS> FRAM64V;

// Option: SPI FRAM using Adafruit library
Adafruit_FRAM_SPI adaFram = Adafruit_FRAM_SPI(FRAM_CS);  // use hardware SPI

//------------------------------------------------------------------------------
/*
 * There are many designs that could be used to cycle through and print the
 * FRAM memory. The aDumpFram design reads and prints 8 byte groups for a clean display.
 *  Reads a buffer 'aBuf[0...7]' from the FRAM[0...framTotalBytes]
 *      for [0...7] bytes from aBuf
 *        read a byte
 *          print as HEX
 *      next
 *
 *      for [0...7] bytes from aBuf
 *        read a byte
 *          print as DEC
 *      next
 *
 *      for [0...7] bytes from aBuf
 *        read a byte
 *          write as ASCII
 *      next
 *  next buf
 * done at end of framTotalBytes
 *
 */

//------------------------------------------------------------------------------


void aDumpFram() {
  // Use AdaFruit Library to
  // dump the FRAM memory to Serial Monitor

  if (!adaFram.begin()) {
    Serial.print(F("\n *** aDumpFram failed to Initialize ***\n\n"));
  }

  Serial.println(F("\n\n  ** Initialized Adafruit SPI FRAM **"));

  // do some checking and setup...
  bufNum = 0;
  uint32_t startRead = startByte;
  if (framEndByte > framTotalBytes) framEndByte = framTotalBytes; // adjust framEndByte
  if (startRead > 0) {  // starting dump from address other than 0
    bufNum = (startRead / sizeof(aBuf)) ; // calc the bufNum counter for the given startRead
    //       Serial.print("\nstartRead: "); Serial.print(startRead);
    startRead = bufNum * sizeof(aBuf); // adjust startRead to aBuf boundary
    //       Serial.print("\nNew startRead: "); Serial.print(startRead);
    //       Serial.print("\nbufNum: "); Serial.print(bufNum); Serial.print("\n");
  }

  while (startRead < framEndByte) {
    /* 1 */
    // Calc the recPtr to the 8 bytes to be read
    uint32_t recPtr = startRead; // read 8 bytes starting at this FRAM address
    /* debug
    Serial.print("\nstart:"); Serial.print (startRead);
    Serial.print(" bufNum:"); Serial.print (bufNum);
    */
    /* 3 */
    // right justify print HEX start address
    Serial.print("\n 0x");
    if (recPtr < 0x10) Serial.print('0');
    if (recPtr < 0x100) Serial.print('0');
    if (recPtr < 0x1000) Serial.print('0');
    Serial.print(recPtr, HEX);
    Serial.print(": ");

    // read 8 bytes into aBuf
    uint8_t bufPtr = 0;
    for (uint32_t framMemPtr = recPtr; framMemPtr < (recPtr + sizeof(aBuf)); framMemPtr++) { // read 8 bytes from FRAM[]
      aBuf[bufPtr] = adaFram.read8(framMemPtr); // Adafruit read8 one byte
      // write the HEX values
      value = aBuf[bufPtr]; // value of byte from aBuf[0...7]
      if (value < 0x10) Serial.print('0');
      Serial.print(value, HEX); Serial.print(" ");
      bufPtr++;
    }

    /* 4 */
    // right justify print DEC start address
    Serial.print(" d");
    if (framTotalBytes >= 100000) {
      if (recPtr < 100000) Serial.print('0');
    }
    if (framTotalBytes >= 10000) {
      if (recPtr < 10000) Serial.print('0');
    }
    if (framTotalBytes >= 1000) {
      if (recPtr < 1000) Serial.print('0');
    }
    if (framTotalBytes >= 100) {
      if (recPtr < 100) Serial.print('0');
    }
    if (recPtr < 10) Serial.print('0');
    Serial.print(recPtr, DEC);
    Serial.print(": ");

    // write the DEC values on the same line
    // process each byte of 8 byte aBuf
    for (uint8_t bufPtr = 0; bufPtr < sizeof(aBuf); bufPtr++) { // individual bytes from aBuf
      value = aBuf[bufPtr]; // value of byte from aBuf[0...7]
      if (value < 100) Serial.print(' ');
      if (value < 10) Serial.print(' ');
      Serial.print(value, DEC); Serial.print(" ");
    } // for (uint8_t bufPtr = 0; bufPtr < sizeof(aBuf); bufPtr++)

    /* 5 */
    // right justify print ASC start address
    Serial.print(" a");
    if (framTotalBytes >= 100000) {
      if (recPtr < 100000) Serial.print('0');
    }
    if (framTotalBytes >= 10000) {
      if (recPtr < 10000) Serial.print('0');
    }
    if (framTotalBytes >= 1000) {
      if (recPtr < 1000) Serial.print('0');
    }
    if (framTotalBytes >= 100) {
      if (recPtr < 100) Serial.print('0');
    }
    if (recPtr < 10) Serial.print('0');
    Serial.print(recPtr, DEC);
    Serial.print(": ");

    // write the Ascii Text values on the same line
    // process each byte of 8 in aBuf
    for (uint8_t bufPtr = 0; bufPtr < sizeof(aBuf); bufPtr++) { // individual bytes from aBuf
      value = aBuf[bufPtr]; // value of byte from aBuf[0...7]
      if ((value > 32) && (value < 124)) {
        Serial.write(value); Serial.print(" ");
      } else {
        Serial.print("."); Serial.print(" ");
      }
    } //for (uint8_t bufPtr = 0; bufPtr < sizeof(aBuf); bufPtr++)
    // increment bufNum (0...framTotalBytes/sizeof(aBuf) )
    //Serial.print("\nstart:"); Serial.print (startRead);
    //Serial.print(" bufNum:"); Serial.print (bufNum);
    bufNum++;
    startRead = startRead + sizeof(aBuf);
  } // while (startRead <= framEndByte) {
} // end aDumpFram()


//------------------------------------------------------------------------------

//------------------------------------------------------------------------------
/*
 * There are many designs that could be used to cycle through and print the
 * FRAM memory. The rDumpFram design prints 8 byte groups for a clean display.
 *  Reads a buffer 'buf[0...63]' from the FRAM[0...framTotalBytes]
 *    for [0...7] blocks (cycle 8 times)
 *      for [0...7] bytes
 *        read a byte
 *          print as HEX
 *      next
 *
 *      for [0...7] bytes
 *        read a byte
 *          print as DEC
 *      next
 *
 *      for [0...7] bytes
 *        read a byte
 *          write as ASCII
 *      next
 *    next block
 *  next buf
 * done at end of framTotalBytes
 *
 */

//------------------------------------------------------------------------------

void rDumpFram() {
  // Use RamDisk Library to
  // dump the FRAM memory to Serial Monitor

  FRAM64V.begin();

  Serial.println(F("\n\n  ** Initialized RamDisk SPI FRAM **"));

  // do some checking and setup...
  bufNum = 0;
  uint32_t startRead = startByte;
  if (framEndByte > framTotalBytes) framEndByte = framTotalBytes; // adjust framEndByte
  if (startRead > 0) {  // starting dump from address other than 0
    bufNum = (startRead / sizeof(buf)) ; // calc the bufNum counter for the given startRead
    //       Serial.print("\nstartRead: "); Serial.print(startRead);
    startRead = bufNum * sizeof(buf); // adjust startRead to buf boundary
    //       Serial.print("\nNew startRead: "); Serial.print(startRead);
    //       Serial.print("\nbufNum: "); Serial.print(bufNum); Serial.print("\n");
  }

  while (startRead < framEndByte) {
    /* 1 */
    // read 64 bytes
    FRAM64V.read(startRead, buf, sizeof(buf));
    // debug
    //Serial.print("\nbufNum: "); Serial.print(bufNum); Serial.print("\n");

    /* 2 */
    // process each of 64 bytes in buf
    //    process each [block of 8 bytes] for print width, 8 times
    //      for each block of 8 bytes in buf
    for (uint8_t iBlk = 0; iBlk < 8; iBlk++) { // block iBlk of 8 bytes from buf
      uint16_t recPtr = (bufNum * sizeof(buf)) + (iBlk * 8); // iBlk[0...7]*8

      /* 3 */
      //---------------------------------------------
      // write the HEX values
      // print iBlk start address as HEX
      Serial.print("\n 0x");
      if (recPtr < 0x10) Serial.print('0');
      if (recPtr < 0x100) Serial.print('0');
      if (recPtr < 0x1000) Serial.print('0');
      Serial.print(recPtr, HEX);
      Serial.print(": ");

      // write the HEX values
      // process each byte of 8 bytes from 8-byte block from buf to make 8-byte print record
      for (uint8_t j = 0; j < 8; j++) { // j individual bytes from block iBlk
        uint16_t bufPtr = (iBlk * 8) + j; // iBlk[0...63]*8 + j[0...7]
        value = buf[bufPtr]; // value of byte from buf[0...63]
        //        if needed, the FRAM memory address is framMemPtr
        //        uint16_t framMemPtr = (bufNum * 64) + bufPtr; // bufNum[0...128]*64 + iBlk[0...63]*8 + j[0...7]
        //        uint8_t value2 = fram.read8(framMemPtr); // Adafruit read8
        /* debug
          Serial.print("start:"); Serial.print (startRead);
          Serial.print(" bufNum:"); Serial.print (bufNum);
          Serial.print(" iBlk:"); Serial.print (iBlk);
          Serial.print(" j:"); Serial.print (j);
          Serial.print(" bufPtr:"); Serial.print (bufPtr, HEX); // buf[0...63] (0-3F)
          Serial.print(" framMemPtr:"); Serial.print (framMemPtr, HEX);
          Serial.print(" v:"); Serial.print (value, HEX);
          Serial.print(" v2:"); Serial.print (value2, HEX);
          Serial.print(""); Serial.print("");
        */
        if (value < 0x10) Serial.print('0');
        Serial.print(value, HEX); Serial.print(" ");
      } // for (uint8_t j = 0; j < 8; j++) {

      /* 4 */
      //---------------------------------------------
      // write the DEC values on the same line
      // print iBlk start address as DEC
      Serial.print(" d");
      if (framTotalBytes >= 100000) {
        if (recPtr < 100000) Serial.print('0');
      }
      if (framTotalBytes >= 10000) {
        if (recPtr < 10000) Serial.print('0');
      }
      if (framTotalBytes >= 1000) {
        if (recPtr < 1000) Serial.print('0');
      }
      if (framTotalBytes >= 100) {
        if (recPtr < 100) Serial.print('0');
      }
      if (recPtr < 10) Serial.print('0');

      Serial.print(recPtr, DEC);
      Serial.print(": ");

      // process each byte of 8 bytes from 8-byte block from buf to make 8-byte print record
      for (uint8_t k = 0; k < 8; k++) { // j individual bytes from block iBlk
        uint16_t bufPtr = (iBlk * 8) + k; // iBlk[0...63]*8 + j[0...7]
        value = buf[bufPtr]; // value of byte from buf[0...63]
        //Serial.print("  ");
        if (value < 100) Serial.print(' ');
        if (value < 10) Serial.print(' ');
        Serial.print(value, DEC); Serial.print(" ");
      } // for (uint8_t k = 0; k < 8; k++) {

      /* 5 */
      //---------------------------------------------
      // write the ASC values
      // print iBlk start address as ASC
      Serial.print(" a");
      if (framTotalBytes >= 100000) {
        if (recPtr < 100000) Serial.print('0');
      }
      if (framTotalBytes >= 10000) {
        if (recPtr < 10000) Serial.print('0');
      }
      if (framTotalBytes >= 1000) {
        if (recPtr < 1000) Serial.print('0');
      }
      if (framTotalBytes >= 100) {
        if (recPtr < 100) Serial.print('0');
      }
      if (recPtr < 10) Serial.print('0');
      Serial.print(recPtr, DEC);
      Serial.print(": ");

      // write the Text values on the same line
      // process each byte of 8 bytes from 8-byte block from buf to make 8-byte print record
      //Serial.print("  a> ");
      for (uint8_t l = 0; l < 8; l++) { // j individual bytes from block iBlk
        uint16_t bufPtr = (iBlk * 8) + l; // iBlk[0...63]*8 + j[0...7]
        value = buf[bufPtr]; // value of byte from buf[0...63]
        if ((value > 32) && (value < 124)) { // write ASC characters from 33 through 123
          Serial.write(value); Serial.print(" ");  // a space is placed between each ASC value for clarity
        } else {
          Serial.print("."); Serial.print(" "); // a dot is used for non-print ASC values
        }
      } // for (uint8_t l = 0; l < 8; l++) {

    } //for (uint8_t iBlk = 0; iBlk < 8; iBlk++) {
    // increment bufNum (0...framTotalBytes/sizeof(buf) )
    bufNum++;
    //if (startRead < framEndByte) startRead = startRead + sizeof(buf);
    startRead = startRead + sizeof(buf);
  } // while (startRead <= framTotalBytes) {
} // end rDumpFram()


//------------------------------------------------------------------------------
//------------------------------------------------------------------------------

void setup() {
  Serial.begin(115200);
  //Serial.println(F("Type any character to begin"));
  Serial.println(F("Begin ADX_dumpFram_01 >"));
  //while (!Serial.available());

  aDumpFram(); // print using Adafruit FRAM Library

  //rDumpFram(); // print using RamDisk Library

  Serial.print("\nDone\n");
}

//------------------------------------------------------------------------------
//------------------------------------------------------------------------------

void loop() {}

//------------------------------------------------------------------------------
//------------------------------------------------------------------------------

24LC16B EEPROM
Ref: http://www.microchip.com/wwwproducts/en/24LC16B
The Microchip 24LC16B EEPROM memory chip was also on hand for testing. The 24LC16B, also used with PICAXE, is an I2C device with 8 Blocks of 256 bytes of memory. Each Block is separately addressable as part of the slave address, and each of the 256 bytes of the Block are addressed as a starting byte address and string of bytes not to exceed the Block (page buffer) length (256 bytes). Only one 24LC16B can be used on a single I2C bus. Larger 4K-128K 24LCxx EEPROMs that use word addressing, can use up to 4 to 8 devices on a single I2C bus. Since the block size of the 24LC16B is 256 bytes, it could easily be used to read/write 8 different text/data strings of 0-255 bytes for applications that require extra dedicated memory. The FRAM, on the other hand, has contiguous 8,192 to 262,144 bytes of storage that can be accessed in one or many byte increments. The RamDisk software adds a FAT style file system using solid state memory storage.
What Was Learned?
There are several different memory options that can be used with Arduino, and software libraries have been written that can take advantage of the differing types of memory hardware, using both SPI and I2C. The Arduino 328P processor dynamic program memory can not be expanded, so using a 16-bit or 32-bit processor with larger program (flash) memory might be required. The cursory testing described above did not require high speeds, which may be a requirement for some applications and would limit what memory chips might provide the required speed.
The Arduino 328P begins to run out of I2C or SPI address pins fairly rapidly, especially when adding multiple sensors for data collection. A port expander (MCP23S17, CD74HC4067, or PCF8574) chip provides a means for increasing the number of Arduino pins for additional address ports to resolve this issue.
FRAM memory was quite versatile and relatively easy to add in code to Arduino sketches. The basic knowledge gained while testing the above memory chips provides some insight into using external memory in Arduino apps.

(Apr 8, 2016)

Saturday, April 2, 2016

YellowJacket Arduino with WiShield Wifi (Part 16)

TCP vs UDP — 
Previous examples in this series have used TCP to transmit data between YellowJacket Arduino’s and Macs over wifi using the WiShield software. In this post, UDP will be examined and a simple example will be developed using the WiShield software to determine how the software library works using UDP.
Ref:http://electronics.stackexchange.com/questions/30474/udp-vs-tcp-protocols
Quoting Ref: “TCP and UDP are protocols on top of the IP layer. TCP and UDP differ in the strategies they use to get data to the application.
TCP is a connection-oriented protocol. When the two sides connect, they exchange a sequence number and start counting bytes sent back and forth. All bytes sent back and forth are counted within the sequence number. So one side can say “I believe I sent you bytes 20-40 in that last packet, can you confirm that you received those bytes in that packet?” This allows each side to know if data wasn’t received or was received out of order, and parts of the protocol specify how each side rectifies the situation (ie, with re-requests for certain packets). There are timeouts as well that determine whether the connection has become interrupted. If a connection is lost like this, a new session has to be re-negotiated and the sequential count starts over (but not necessarily from byte 0 – a certain amount of randomness is required to prevent some attacks). Practically this means that a connection like this can’t just be picked up again once it becomes interrupted – you have to re-establish the connection. There are also flow-control features that ensure data isn’t sent too fast for one side to process. TCP is very useful where the data absolutely must get through despite delays.
UDP is different. There is no connection, no sequential numbers, no timeouts. Packets are sent between the two sides with no other necessary information. Since there is no connection the packets can be sent at any time. There’s also no guarantee that data gets through. If you send a packet, it may not get there and you’d never know. Typically, the data is sent with checksums, sequence numbers and such into the data you send if it’s necessary for a more secure link. Because of the lack of timeouts and retries it’s much faster than TCP. Typically UDP is used for situations where speed is more important that getting all of the data.
When data is sent via UDP, one of two things will happen: either a packet will reach its destination within a moderately-short time, or it won’t. The sender will be notified that the packet was started on its journey, whether or not it actually reached its destination. If the packet is not delivered successfully, the receiver will never know of its existence. It’s important to note that while UDP will generally deliver packets in reasonably-timely fashion, there is no guarantee that packets will reach their destination in the order they were sent. Most of the time they will, but some networking equipment may arbitrarily divide up the packets going through them among several different links, some of which might be slightly faster than others.
TCP, unlike UDP, establishes a connection between two parties and attempts to ensure that all information supplied by each side is received by the other, in the order that it was transmitted; it guarantees that either every byte will be delivered in sequence, or the connection will be declared invalid. The simplest way to think of TCP is as having nodes send messages of the following format:
I have received information up to but not including your byte #1253, …
… and my data, starting at byte #4381, is “QUACK”.
Each node sends a message of the above format when either it has data that the other node has not acknowledged, or when it has received data that it has not yet acknowledged. One very nice thing about TCP is that it doesn’t matter too much if an acknowledgment gets lost. If data is received successfully but the transmitter doesn’t know about it, the transmitter will include redundant data the next time it decides to transmit, but the receiver will simply ignore that redundant data and acknowledge all the useful data it received.
A few things to note about TCP:
  1. A device which transmits information will expect an acknowledge, and will attempt retransmission if it doesn’t get one. If it persistently fails to get an acknowledge, it will decide the connection is dead, but the other party to the connection may never know about it.
  2. TCP generally only sends packets when at least one party to the communication has “something to say”. If neither node has anything to say, a TCP connection could sit idle for hours or days without any data flowing through.
  3. Some TCP implementations will, if no data flows for an extended period of type (often somewhere between a minute and an hour), resend their last byte of data in what’s called a “keepalive” packet, “pretending” that they didn’t receive an acknowledgement for it. In the normal chain of events, the receiver will know that it already received that byte, and thus discard it, but they will also know that the transmitter is probably expecting an acknowledgment for it. If the transmitter that sent the “keepalive” doesn’t get a response, it will know the other party to the connection has gone missing.
  4. A device which transmits via TCP will know that all of its data has been successfully delivered. A device which is receiving TCP will not know when it has received all data that has been sent to date, unless it receives a packet with a “FIN” flag, indicating that there will be no further data.”
Ref: http://stackoverflow.com/questions/12251551/send-arduino-data-via-a-wi-fi-shield-to-a-specific-ip-address-on-the-lan — This Ref responds to a user wanting to create a UDP app for the Copperhead WiShield Wi-Fi Server.
Ref: https://www.arduino.cc/en/Tutorial/UDPSendReceiveString, and http://duino4projects.com/sending-and-receiving-string-via-udp-using-arduino/ — These Refs by Michael Margolis use the Arduino and Ethernet boards to communicate with a Processing sketch running on a PC/Mac.
Getting UDP Working in WiShield
The following code has been adapted to use the YellowJacket Wifi to communicate with the UDPSendReceiveString Processing sketch (above Ref) running on a Mac. This example uses UDP to send text strings back and forth between the two processors, but was most useful for determining how to get UDP working with the WiShield software. The first step taken was to get a UDP signal sent from the Processing app on the Mac, through wifi, to the YellowJacket Arduino running the WiShield software. The use of the Wireshark app running on the Mac and finding the UDP messages that were being sent to the YJ Arduino at ip 10.0.1.57 was critical to getting things working. The vanilla WiShield software does not have any Serial Monitor debugging to allow tracking of UDP signals. After a lot of tracing and debugging, the proper port assignments turned out to be essential (as would be expected) but there was no obvious documentation of where those port assignments should be made. It turns out to be fairly straight forward once you know where to look and make changes.
In the Processing sketch, the WiShield client ip address is set to 10.0.1.57 with the client UDP port 1026. The Mac server wifi ip address assigned by the local network was 10.0.1.11, and the sketch assigned server UDP port 6000.
When the Processing app runs, a small Java terminal window opens, which will send out “Hello” to ip 10.0.1.57, port 1026, each time there is a keystroke in the terminal window. With the YJ57 not on, Wireshark sensed a signal to “Broadcast” and info = “Who has 10.0.1.57? Tell 10.0.1.11″, so the Processing signal was trying to be sent.
The next task was to get UDP working in a sketch on the YellowJacket. The vanilla UDPApp example code was edited to use the local network wifi settings as in previous examples, and setup() included ” WiFi.init();” and loop() contained ” WiFi.run();”. In apps.conf.h, “#define APP_UDPAPP” must be uncommented, and in uip-conf.h, “#define UIP_CONF_UDP 1” must be 1 rather than 0 (TCP) to allow UDP to run in WiShield. Running that sketch on the YellowJacket, and the Processing app on the Mac, showed “Source: 10.0.1.11; Destination: 10.0.1.57; Protocol: UDP; Info: 6000 –> 1026 Len=5” in Wireshark, which indicated that UDP was communicating over wifi since 10.0.1.57 was connected to the local network through wifi, and the Processing app was sending out the “Hello” packets to 10.0.1.57. However, 10.0.1.57 was not responding back to the Processing app, since the YJ57 ports had not been assigned.
The UDPApp.ino sketch has the udpapp.c in a separate tab. The udpapp_init module in udpapp.c must be modified to assign the correct ip and ports:
        // originalcode:
 uip_ipaddr(&addr, 192,168,1,100);
 c = uip_udp_new(&addr, HTONS(0));
 if(c != NULL) {
  uip_udp_bind(c, HTONS(12344));

        // modified code:
 uip_ipaddr(&addr, 10,0,1,11); // (server ip)
 c = uip_udp_new(&addr, HTONS(6000)); // (server port)
 if(c != NULL) {
  uip_udp_bind(c, HTONS(1026)); // (YJ57 client port)
With the modified UDPApp.ino running, Wireshark displayed the data flowing back and forth between the two devices using wifi. The Arduino Serial Monitor window showed no indication of UDP communication. Fixing that became the next task.
No. Time        Source      Destination  Protocol Length Info                 Data
15  7.912052    10.0.1.11   10.0.1.57    UDP      47     6000 → 1026  Len=5   Hello
16  7.918114    10.0.1.57   10.0.1.11    UDP      68     1026 → 6000  Len=26  Hello. What is your name?
17  7.919292    10.0.1.11   10.0.1.57    UDP      45     6000 → 1026  Len=3   Mac
18  7.925933    10.0.1.57   10.0.1.11    UDP      58     1026 → 6000  Len=16  Hello Mac
19  7.926600    10.0.1.11   10.0.1.57    UDP      45     6000 → 1026  Len=3   Mac
32  19.069058   10.0.1.11   10.0.1.57    UDP      47     6000 → 1026  Len=5   Hello

Processing Sketch on Mac
The processing sketch can be loaded and compiled in the Processing app (Ref: https://processing.org/tutorials/). The UDPSendReceive_P3.pde code is below. The “import hypermedia.net.*;” is required and was found for download at Ref: https://forum.processing.org/two/discussion/9802/looking-for-udp-library-hypermedia-library-dead, and http://ubaa.net/shared/processing/udp/udp.zip.
/*
 * UDPSendReceive_P3
 * Processing sketch to run with this example
 * =====================================================
 * 
 * // Processing UDP example to send and receive string data from Arduino
 * // press any key to send the "Hello Arduino" message
 * 
 * Processing sketch output:
 * send=> Hello
 * Hello. What is your name?
 * send=> Mac
 * Hello, Mac
 * 
 * 
 * Arduino sketch output:
 * === Initializing WiShield as UDP Client on Serial Monitor ===
 * 
 * Server port:6000; Server ip:10,0,1,11
 * Client port:1026; Client ip:10,0,1,57
 * 
 * WiFi Init OK; Start UDP Listening...
 * 
 * Server port:6000; Server ip:10,0,1,11
 * Client port:1026; Client ip:10,0,1,57
 * 
 * Incoming Message Length: 5; Message: Hello (from Processing app)
 * Outgoing Request Message Length: 26; Message: Hello. What is your name?
 * Incoming Message Length: 3; Message: Mac (from Processing app not displayed)
 * Outgoing Response Message Length: 12; Message: Hello, Mac
 * 
 */

import hypermedia.net.*;

UDP udp;  // define the UDP object
  String ip       = "10.0.1.57"; // the remote IP address
  int port        = 1026;    // the destination port


void setup() {
  udp = new UDP( this, 6000 );  // create a new datagram connection on port 6000
  // udp.log( true );           // <-- printout the connection activity
  udp.listen( true );           // and wait for incoming message
}

void draw()
{
}

void keyPressed() {
  
  //String ip       = "10.0.1.57"; // the WiShield remote IP address
  //int port        = 1026;    // the WiShield destination port
  //Wireshark Mac port 6000 to port 1026 on YJ57 10.0.1.57
  //6613  3.784864  10.0.1.11  10.0.1.57  UDP  47  6000 → 1026  Len=5

  udp.send("Hello", ip, port );   // the message to send
  //for (int i=0; i<5; i++) {
  //    udp.send("Hello", ip, port );   // the message to send
  //   for (int j=0; j<10; j++) print (".");
  //}
}

void receive( byte[] data ) {      // <-- default handler
  //void receive( byte[] data, String ip, int port ) { // <-- extended handler

  for (int i=0; i < data.length; i++)
    print(char(data[i]));
  println();
  
  // reply with name
  // send here sends the name but that kills the communication
    udp.send("Mac", ip, port );   // the message to send

}

/* ============================= */

Wireshark showed that the sketches could communicate using wifi, so that part worked as desired. The YellowJacket Wifi sketch and WiShield Library required fairly extensive modifications to display the data being transferred back and forth on the Arduino Serial Monitor. Edits were required in config.h (renamed config_udp.h to indicate that edits had been made in that file), and udpapp.c.
After a click and <cr> on the Processing app window, the Serial Monitor displayed:
=== Initializing WiShield as UDP Client on Serial Monitor ===

Server port:6000; Server ip:10,0,1,11
Client port:1026; Client ip:10,0,1,57

WiFi Init OK; Start UDP Listening...

Server port:6000; Server ip:10,0,1,11
Client port:1026; Client ip:10,0,1,57
Incoming Message Length: 5; Message: Hello (from Processing app)
Outgoing Request Message Length: 26; Message: Hello. What is your name?
    [ not displayed Incoming Message Length: 3; Message: Mac (from Processing app) ]
Outgoing Response Message Length: 10; Message: Hello, Mac

Quite a bit of time was invested in getting UDP working, primarily due to the lack of documentation for implementing UDP on WiShield. UDP will be revisited at a later date, to make it more useful for sending and receiving data.

(Apr 2, 2016)