Package ai.djl.engine.rpc
Class RpcTranslator<I,O>
java.lang.Object
ai.djl.engine.rpc.RpcTranslator<I,O>
- All Implemented Interfaces:
NoBatchifyTranslator<I,,O> PostProcessor<O>,PreProcessor<I>,Translator<I,O>
The
Translator for model deploy on remote model server.-
Constructor Summary
ConstructorsModifierConstructorDescriptionprotectedRpcTranslator(RpcClient client, TypeConverter<I, O> converter) Constructs aRpcTranslatorinstance. -
Method Summary
Modifier and TypeMethodDescriptionprocessInput(TranslatorContext ctx, I input) Processes the input and converts it to NDList.processOutput(TranslatorContext ctx, NDList list) Processes the output NDList to the corresponding output object.Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitMethods inherited from interface ai.djl.translate.NoBatchifyTranslator
getBatchifierMethods inherited from interface ai.djl.translate.Translator
batchProcessInput, batchProcessOutput, getExpansions, prepare
-
Constructor Details
-
RpcTranslator
Constructs aRpcTranslatorinstance.- Parameters:
client- aRpcClientconnects to remote model serverconverter- aTypeConverterto convert data type
-
-
Method Details
-
processInput
Processes the input and converts it to NDList.- Specified by:
processInputin interfacePreProcessor<I>- Parameters:
ctx- the toolkit for creating the input NDArrayinput- the input object- Returns:
- the
NDListafter pre-processing - Throws:
IOException
-
processOutput
Processes the output NDList to the corresponding output object.- Specified by:
processOutputin interfacePostProcessor<I>- Parameters:
ctx- the toolkit used for post-processinglist- the output NDList after inference, usually immutable in engines like PyTorch. @see Issue 1774- Returns:
- the output object of expected type
- Throws:
TranslateException
-