amazon web services - invoke_model of boto3 is not accepting the parameter explicitPromptCaching as one of its arguements - Stac

I am using invoke_model function of boto3 library for aws bedrock for Claude sonnet 3.5 v2, i am trying to use prompt caching by using the InvokeModel API of bedrock for claude, according to the following documentation: PROMPT Caching using bedrock invoke_model must be accepting the parameter explicitPromptCaching, but upon using the latest version of boto3: 1.36.3 at this time, i am getting this error: botocore.exceptions.ParamValidationError: Parameter validation failed: Unknown parameter in input: "explicitPromptCaching", must be one of: body, contentType, accept, modelId, trace, guardrailIdentifier, guardrailVersion, performanceConfigLatency

Here is the function call:

response = self.client.invoke_model(body=body, modelId=self.modelId,explicitPromptCaching = 'enabled')

Also according to another documentation of boto3 in python: boto3 docs invoke_model is not supposed to be accepting the parameter in question. Also upon going through the docs of InvokeModel API, there is nothing about this parameter.

InvokeModel API Docs

To use prompt caching the only option that i may have is ConverseAPI, But my backend is already written using InvokeModel, the advantage of using InvokeModel over ConverseAPI is that the latter uses a unified format of request and response for all the models, this might seem a good use but for me it is not. InvokeAPI is specific to the model vendor you are using, this way you can follow the extensive docs of the vendor model, But for the latter you will have to cope with the incomplete and bizzare docs of aws bedrock and be on their mercy when something goes wrong.

What am i doing wrong? Help is appreciated.

I am using invoke_model function of boto3 library for aws bedrock for Claude sonnet 3.5 v2, i am trying to use prompt caching by using the InvokeModel API of bedrock for claude, according to the following documentation: PROMPT Caching using bedrock invoke_model must be accepting the parameter explicitPromptCaching, but upon using the latest version of boto3: 1.36.3 at this time, i am getting this error: botocore.exceptions.ParamValidationError: Parameter validation failed: Unknown parameter in input: "explicitPromptCaching", must be one of: body, contentType, accept, modelId, trace, guardrailIdentifier, guardrailVersion, performanceConfigLatency

Here is the function call:

response = self.client.invoke_model(body=body, modelId=self.modelId,explicitPromptCaching = 'enabled')

Also according to another documentation of boto3 in python: boto3 docs invoke_model is not supposed to be accepting the parameter in question. Also upon going through the docs of InvokeModel API, there is nothing about this parameter.

InvokeModel API Docs

To use prompt caching the only option that i may have is ConverseAPI, But my backend is already written using InvokeModel, the advantage of using InvokeModel over ConverseAPI is that the latter uses a unified format of request and response for all the models, this might seem a good use but for me it is not. InvokeAPI is specific to the model vendor you are using, this way you can follow the extensive docs of the vendor model, But for the latter you will have to cope with the incomplete and bizzare docs of aws bedrock and be on their mercy when something goes wrong.

What am i doing wrong? Help is appreciated.

• amazon-web-services
• caching
• prompt
• amazon-bedrock
• claude

Share Improve this question Follow asked Jan 22 at 21:51 Anchit RanaAnchit Rana 1111 bronze badge 1

• The first line of the documentation: "Amazon Bedrock prompt caching is currently only available to a select number of customers. To learn more about participating in the preview, see Amazon Bedrock prompt caching." You need to request access to the preview, and be accepted. Or wait for general availability. – Anon Coward Commented Jan 22 at 22:05

Add a comment  | 

1 Answer 1

Sorted by: Reset to default -1

You are in a common issue with bedrock and prompt caching... the explicitPromptCaching parameter is not directly supported within the invoke_model API call itself... it's handled totally by bedrock's backend when some conditions are met, not via a special parameter you set in your code

the documentation can be misleading you don't need to enable prompt caching in the invoke_model request... bedrock handles caching based on the request body (your prompt) if you send the same prompt multiple times bedrock will likely serve the cached response

focus on your prompts are identical for later requests if you want caching... variations in whitespace or minor textual differences will result in cache misses... you don't need to switch to the converse API for basic prompt caching with invoke_model