Package ai.djl.engine.rpc

Class RpcTranslator<I,O>

java.lang.Object

ai.djl.engine.rpc.RpcTranslator<I,O>

All Implemented Interfaces:

NoBatchifyTranslator<I,O>, PostProcessor<O>, PreProcessor<I>, Translator<I,O>

public class RpcTranslator<I,O>
extends Object
implements NoBatchifyTranslator<I,O>

The Translator for model deploy on remote model server.

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	RpcTranslator(RpcClient client, TypeConverter<I,O> converter)	Constructs a RpcTranslator instance.

Method Summary

Modifier and Type	Method	Description
NDList	processInput(TranslatorContext ctx, I input)	Processes the input and converts it to NDList.
O	processOutput(TranslatorContext ctx, NDList list)	Processes the output NDList to the corresponding output object.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface ai.djl.translate.NoBatchifyTranslator

getBatchifier

Methods inherited from interface ai.djl.translate.Translator

batchProcessInput, batchProcessOutput, getExpansions, prepare

Constructor Details

RpcTranslator

protected RpcTranslator(RpcClient client,
 TypeConverter<I,O> converter)

Constructs a RpcTranslator instance.

Parameters:

client - a RpcClient connects to remote model server

converter - a TypeConverter to convert data type

Method Details

processInput

public NDList processInput(TranslatorContext ctx,
 I input)
                    throws IOException

Processes the input and converts it to NDList.

Specified by:

processInput in interface PreProcessor<I>

Parameters:

ctx - the toolkit for creating the input NDArray

input - the input object

Returns:

the NDList after pre-processing

Throws:

IOException

processOutput

public O processOutput(TranslatorContext ctx,
 NDList list)
                throws TranslateException

Processes the output NDList to the corresponding output object.

Specified by:

processOutput in interface PostProcessor<I>

Parameters:

ctx - the toolkit used for post-processing

list - the output NDList after inference, usually immutable in engines like PyTorch. @see Issue 1774

Returns:

the output object of expected type

Throws:

TranslateException