Add support for downloading inference script and code (entry_point) from s3

[urirosenberg]

Describe the feature you'd like
The sagemaker-train-toolkit has the functionality to copy the required code into the container. This is not present in the sagemaker inference toolkit.

How would this feature be used? Please describe.
Customers want to use a generic Docker image in multiple train-inference pipelines.

[billgivzey]

...it's not? man that makes this totally worthless for me then

[admivsn]

...it's not? man that makes this totally worthless for me then

During training the code is uploaded to S3 and then during inference this code would be used for inferences from the trained model.

The problem occurs when we might want to update some inference code (e.g. predict_fn) without retraining the model.

We have a workaround whereby you can include an env variable s3_path containing inference code as an env variable to the inference job, and we use the training toolkit to download and extract the contents into the container before starting the inference server.

# Code to download and extract model code from s3_path before starting model inference server

import os

from sagemaker_inference import model_server
from sagemaker_training import files, environment

# Download code into container
files.download_and_extract(uri=os.environ['s3_path'], path=environment.code_dir)

# Start the model server
model_server.start_model_server()

# Upload model code (containing inference code) to use without retraining model

from sagemaker.fw_utils import tar_and_upload_dir

source_code_s3_path, _ = tar_and_upload_dir(...)

# Use new source code to do inference
from sagemaker.transformer import Transformer

transformer = Transformer(..., env={"s3_path": source_code_s3_path})
transformer.transform(...)