YellowJacket Arduino with WiShield Wifi (Part 14)

More About Handling JSON Data and Using POSTrequest —

In Part 13, the YJ56 Server sketch responded to “http://10.0.1.56/json” by returning a JSON-formatted data packet, such as, “{“tSensor”:”2016-03-26 14:55:29″,”tempF”:”79.1″,”light”:”281″,”button”:”1″,”led”:”1″}”. The JSON data packet was returned as part of the html stream but enclosed in non-standard <json> and </json> tags. These tags are ignored by an Arduino sketch and WiShield wifi software and can be used as delimiters to parse out the JSON data packet on the Client. The parsed JSON data can then be repackaged and sent to an Apache/php server that can send the data to mySQL for writing to a data table. The ArduinoJson parsing routines (Ref: https://github.com/bblanchon/ArduinoJson/wiki) are well done and saved a lot of time in developing the sketch code used by the YJ57 client.

The YJ56 Sensor Data Server as described, had a time clock, temperature sensor, photocell light sensor, a Pin 3 INT1 On/Off switch sensor, and an led On/Off display. The http code that selects which action to take was based on Ref: http://www.dfrobot.com/forum/viewtopic.php?f=2&t=61. The YJ56 Server can receive remote http commands to set or clear the On/Off switch sensor and the led On/Off display, or they could be set by the YJ56 server hardware or software. Also, both the switch sensor and led can be turned on or off with http commands from a web browser. Those settings are then transmitted within the JSON data to the data base. Some action could then be taken (i.e., alert, turn something on/off) depending on the user’s requirements. The intent of this part of the code is to demonstrate simple methods of remotely checking on sensor settings, such as, has a door been opened, or to turn On/Off a light or valve. The code does not have any security, but it would work in a local, secure wifi situation.

Another issue with using JSON parsing on the Arduino-328P is available dynamic memory which is limited to 2,048 bytes. One goal was to write the JSON data to an SD Card file, but the SD Card library could not be loaded with the ArduinoJson library and compiled within the 2,048 byte dynamic memory limit. Options might be to use an Arduino with more dynamic memory, add a third Arduino that handled the SD Card storage function, or to add other out-board SRAM or FRAM memory chips that might store the data without requiring the large memory requirements of a file system. For the time being, the data were shunted off to Apache/php/mySQL which relieved the dynamic memory issues and provided a method for sending the data to a data base server that has extensive capabilities. This approach also opens the possibilities for using multiple Arduino Data Servers transmitting data to a centralized data base server.

WiShield – Where is the http code?

In several Arduino wifi libraries, the html code for Get and Post requests is added as part of the sketch code. In WiShield, the html code statements are defined in strings.h. Only Get Content-Type:/form is defined in WiShield. For JSON Post Requests, the strings.h file was modified to add a /json Content-Type, as follows:

// Content type form data
const char PROGMEM contentTypeForm[] = {"Content-Type: application/x-www-form-urlencoded"};

// ref: http://stackoverflow.com/questions/4794975/problem-creating-a-http-post-message-with-the-wiserver-library
// add Content-Type /json
const char PROGMEM contentTypeJson[] = {"Content-Type: application/json; charset=utf-8"};

To choose which Content-Type to use in a sketch, a variable, “fType”, was added to WiShield. WiServer.h and .cpp were renamed to WiServerIO.h and .cpp to clarify that changes had been made to those files. Also, all references to WiServer.h had to be changed in the library files and sketches to use WiServerIO.h. In WiServerIO.cpp, a new contentTypeJson was added:

extern const char PROGMEM contentTypeForm[];
extern const char PROGMEM contentTypeJson[];

And below the extern definitions, the fType variable and setContentType function were added:

// use fType to select contentType
int fType = 0;
void setContentType(int value){ fType = value; }

Also in WiServerIO.cpp, code was added within “if (isPost){” to respond to the setting of “fType”:

if (isPost) {
  // Since a post has a body after the blank header line, it has to include
  // an accurate content length so that the server knows when it has received
  // all of the body data.
  char* lengthFieldPos; // Cursor position where the content length place holder starts
  char* contentStart; // Start of the body
  char* contentEnd; // End of the body

    // use 0=text or 1=json data format; added for JSON POST
    if (fType > 0) {
        // JSON data
        WiServer.println_P(contentTypeJson);
    }else{
        // TEXT form data
        WiServer.println_P(contentTypeForm);
}

In WiServerIO.h, add the function definition:

void setContentType(int value); // choose 0=text or 1=json data

Finally, within the Arduino sketch “void setup(){“, the fType can be chosen as:

// choose text or json format data; sets value in WiServerIO.h, .cpp
setContentType(1);     // 0=Text, 1=JSON

setContentType(0) is the default set in WiServerIO.cpp, so sketches using standard html Get requests do not need the setContentType(0) statement added. The code modifications above, can be also extended for transmitting other data types, such as, image files. Rather than using integers to choose the Type, SetContentType might better be defined as a char array that would allow the user to define the desired content type in the sketch, such as,

setContentType("Content-Type: application/json; charset=utf-8");

Ref: http://forum.arduino.cc/index.php?topic=49407.0 shows sketch examples of many data types which might be considered.

• Content-Type: text/html
• Content-Type: text/css
• Content-Type: image/png
• Content-Type: image/jpeg
• Content-Type: image/gif
• Content-Type: video/mpeg
• Content-Type: application/pdf
• Content-Type: application/x-javascript
• Content-Type: application/xml
• Content-Type: text

POSTrequest Example Sketch Code

Following are portions of sketch code that can be used to create a POST request which might be desired for more secure data transmission. See previous WiServer example sketches for mySensor(), printData(), and other sketch code to fill out this example. Notice the three sendMyRequest functions that are defined in WiShield Library request.cpp.

/*************************** createBody function *********************************/

// POSTrequest setBodyFunc(bodyFunction body);
void createBody()
{
  // char json[] = "{\"val1\":\"12.34\",\"val2\":\"56\"}"; // example JSON
  //    Serial.print(F(" json body >")); Serial.print(json); Serial.println(F("< "))
  // WiServer.print(json);

  // place sensor data into JSON format
  WiServer.print("{\"val1\":\"");
  WiServer.print(vj1);
  WiServer.print("\",\"val2\":\"");
  WiServer.print(vj2);
  WiServer.println("\"}");

}
/*************************** end createBody function *********************************/

/*************************** POSTrequest function *********************************/
//
uint8_t request_ip[] = {10, 0, 1, 50}; // Wifi IP Address for requested server Mac
char* myhost = "mac.localhost";
char inCMD[23] = "/write_json_2data.php?"; // php file to process the JSON POSTrequest

// A request that POSTs to a php server at request_ip
// Create an instance of POSTrequest with the JSON text in the body
POSTrequest sendMyRequest(request_ip, 80, myhost, inCMD, createBody);

/*************************** setup function *********************************/
//
void setup() {

  // If you want to set the aref to something other than 5v
  //analogReference(EXTERNAL);  // enable EXTERNAL if using 3.3 AREF
  analogReference(DEFAULT);     // DEFAULT for no external AREF

  Serial.begin(115200);
  delay(10);

  // Connect to WiFi access point.
  Serial.println();
  Serial.print(F("  > Init WiClient to External IO Server: "));
  Serial.println(myhost);

  WiServer.init(NULL);  // use for POSTrequests

  // Ask WiServer to generate log messages (optional)
  WiServer.enableVerboseMode(true);
  
  // choose text or json format data; sets value in WiServerIO.h, .cpp
  setContentType(1);     // 0=Text, 1=JSON
  
  // Have the printData function called when data is returned by the server
  sendMyRequest.setReturnFunc(printData);
  sendMyRequest.setBodyFunc(createBody);

  Serial.println (F(" >> Send data on 30 sec intervals... "));
  Serial.println (F(">>>"));
}
/*************************** end setup function *********************************/

// Time (in millis) when the data should be retrieved
unsigned long updateTime = 0; // tempTime;
unsigned long waitTime = 1000UL * 30UL ; // 30 second cycle

/*************************** loop function *********************************/
//
void loop() {

  // Check if it's time to get an update
  if (millis() >= updateTime)
  {
    // Obtain sensor data as char arrays
    mySensor();

    Serial.println("Start POST >");
    sendMyRequest.submit();
    Serial.println("< End POST");
    // Get another update waitTime from now
    updateTime += waitTime;
  }

  // Run WiServer task
  WiServer.server_task();

  delay(100);
}
/*************************** end loop function *********************************/

And a php file that would process the returned POSTrequest for 2 sensor values must read the JSON input stream and use json_decode to extract the data elements:

<?php

 // File "write_json_2data.php"
 // written as part of Arduino sketch "JsonSend2Values"
 // to insert two values into MySql table using POSTrequest
 // Prepare variables for database connection

 $dbhost = 'localhost:3306';
 $dbuser = "arduino";  // enter database username
 $dbpass = "arduinotest";  // enter database password
 $conn = mysql_connect($dbhost, $dbuser, $dbpass);

 if(! $conn ) {
   die('Could not connect: ' . mysql_error());
 }
 
 mysql_select_db(‘Arduino’);
 
 // For val1,val2 submitted as JSON POSTrequest
        // read the json input stream as 'php://input' from client
 $jsondata = file_get_contents('php://input');
 // echo $jsondata;

        //convert json object to php associative array
        $data = json_decode($jsondata, true);
 // echo $data;
 
 // get the data value details
 $v1 = $data['val1'];
 $v2 = $data['val2'];

 // Prepare the SQL statement
 $sql = "INSERT INTO Arduino.sensor2 (val1,val2) VALUES ('$v1', '$v2')"; 
 //echo $sql;
 
 // Execute SQL statement
 $retval = mysql_query( $sql, $conn );

 if(! $retval ) {
   die('Could not enter data: ' . mysql_error());
 }

 echo "\nEntered 2 data values successfully into Table sensor2\n";

 mysql_close($conn);

?>

(Mar 28, 2016)