Package dev.langchain4j.model.language
Interface TokenCountEstimator
-
- All Implemented Interfaces:
public interface TokenCountEstimatorRepresents an interface for estimating the count of tokens in various text types such as a text, prompt, text segment, etc. This can be useful when it's necessary to know in advance the cost of processing a specified text by the LLM.
-
-
Method Summary
Modifier and Type Method Description abstract intestimateTokenCount(String text)Estimates the count of tokens in the given text. intestimateTokenCount(Prompt prompt)Estimates the count of tokens in the given prompt. intestimateTokenCount(TextSegment textSegment)Estimates the count of tokens in the given text segment. -
-
Method Detail
-
estimateTokenCount
abstract int estimateTokenCount(String text)
Estimates the count of tokens in the given text.
- Parameters:
text- the text.- Returns:
the estimated count of tokens.
-
estimateTokenCount
int estimateTokenCount(Prompt prompt)
Estimates the count of tokens in the given prompt.
- Parameters:
prompt- the prompt.- Returns:
the estimated count of tokens.
-
estimateTokenCount
int estimateTokenCount(TextSegment textSegment)
Estimates the count of tokens in the given text segment.
The metadata will not be included in the estimate.
- Parameters:
textSegment- the text segment.- Returns:
the estimated count of tokens.
-
-
-
-