[EXAMTOPIC] JSON Syntax for AI Platform prediction API

JSON Syntax for AI Platform prediction API

Formatting instances as JSON strings

• Getting online predictions | AI Platform Prediction | Google Cloud
• Prediction overview | AI Platform Prediction | Google Cloud
• Formatting of the request and response body - the prediction request

Prediction input data WITH API CALL

• Batch : Text file with JSON instance strings or TFRecord file (may be compressed)
• Online : JSON request message

Batch

Basic format for both online and batch prediction : a list of instance data tensors.

• Plain lists of values or members of a JSON object - depending on how you configured your inputs in your training application

{"values": [1, 2, 3, 4], "key": 1}

Rules - makeup of JSON strings

1. TOP LEVEL of instance data must be a JSON object - dictionary of key/value pairs
2. Individual values can be strings, numbers, lists. CANNOT EMBED JSON OBJECTs.
3. List must contain ONLY ITEMS OF SAME TYPE. CANNOT MIX STRING & NUMERIC VALUES.

• a single input : DONT NEED TO WRAP IT IN A JSON OBJECT.

# single tensor case 
{"values": [1, 2, 3, 4]} 
# DONT NEED TO USE THIS FORMAT 
[1, 2, 3, 4] # JUST FORMAT EACH INSTANCE AS A LIST

• Image : a 3-dimensional array of 8-bit integers

  {
  "tag": "beach",
  "image": [
    [
      [138, 30, 66],
      [130, 20, 56],
      ...
    ],
    [
      [126, 38, 61],
      [122, 24, 57],
      ...
    ],
        ...
  ]
}

• Binary data : CANT BE formatted as the UTF-8 ENCODED STRING that JSON supports. ** MUST USE BASE64 ENCODING**.

# encoded string must be formatted as a JSON object with a single key named "b64"
{"image_bytes":{"b64": base64.b64encode(jpeg_data)}}

Online ***

Request an online prediction by sending input data instances as a JSON string in a predict

Formatting input for online prediction

Pass input instances for online prediction as the message body for the predict request.

1. Make each instance an item in a list
2. Name the list member "instances".

# Batch Prediction  
{"values": [1, 2, 3, 4], "key": 1}
# Online Prediction  
{"instances": [{"values": [1, 2, 3, 4], "key": 1}]}

• If you don't specify a model version, your prediction request uses the default version of the model.

Q 22.

You trained a text classification model. You have the following SignatureDefs:

You started a TensorFlow-serving component server and tried to send an HTTP request to get a prediction using:
headers = {"content-type": "application/json"} json_response = requests.post('http: //localhost:8501/v1/models/text_model:predict', data=data, headers=headers)
What is the correct way to write the predict request?

• A. data = json.dumps({“signature_name”: “seving_default”, “instances” : [[‘ab’, ‘bc’, ‘cd’]]})
• B. data = json.dumps({“signature_name”: “serving_default”, “instances” : [[‘a’, ‘b’, ‘c’, ‘d’, ‘e’, ‘f’]]})
• C. data = json.dumps({“signature_name”: “serving_default”, “instances” : [[‘a’, ‘b’, ‘c’], [‘d’, ‘e’, ‘f’]]})
• ⭕ D. data = json.dumps({“signature_name”: “serving_default”, “instances” : [[‘a’, ‘b’], [‘c’, ‘d’], [‘e’, ‘f’]]})

KEY - Batch Prediction, Tensorflow Serving

shape (-1,2) : represents a vector with any number of rows but only 2 columns.

• -1: AUTO-EXPAND

• Make a request to your model in TensorFlow Serving