python - How to invoke the Iris endpoint in the sample-notebooks for Amazon SageMaker using SDK


Keywords:python 


Question: 

I'm trying to invoke the endpoint from the tensorflow_iris_dnn_classifier_using_estimators provided in the sample notesbooks. In the sample, it invokes the endpoint using the same python object generated in the deployment process. In a large system, I need to know how to invoke this endpoint without this object and possibly in different languages. This is what I've tried:

import struct
import boto3

client = boto3.client('sagemaker-runtime')

query = [6.4, 3.2, 4.5, 1.5]
buf = struct.pack('%sf' % len(query), *query)

response = client.invoke_endpoint(
    EndpointName='sagemaker-tensorflow-py2-cpu-2018-01-16-18-22-54-458',
    Body=buf
)

What are I doing wrong? I get the following error from cloudwatch:

[2018-01-16 19:51:21,091] ERROR in serving: 'utf8' codec can't decode byte 0xcd in position 0: invalid continuation byte
2018-01-16 19:51:21,091 ERROR - model server - 'utf8' codec can't decode byte 0xcd in position 0: invalid continuation byte
10.32.0.2 - - [16/Jan/2018:19:51:21 +0000] "POST /invocations HTTP/1.1" 500 0 "-" "AHC/2.0"

1 Answer: 

You should be able to call the inference using the InvokeEndpoint in any of the languages that are supported by the AWS SDK for SageMaker.

The Python SDK is what you have in your example above, but a similar syntax can be done in java, JavaScript, .NET and others.

The call in the python API references ( ) is

response = client.invoke_endpoint(
    EndpointName='string',
    Body=b'bytes'|file,
    ContentType='string',
    Accept='string'
)

The Body should be encoded as the model knows how to read. I'm not sure why you decided to encode the way that you did, but you can try simpler options and use the b' prefix. The error message simply says that model was not eable to decode your Body input.