NEW

Join us at SmartCon 2023—where Web3 gets real. Get your ticket.

Using Off-chain Secrets in Requests

As explained in the Using Secrets in Requests guide, the Chainlink Functions starter kit shares encrypted secrets off-chain with a Decentralized Oracle Network (DON) using gists. This tutorial shows you how to share encrypted secrets off-chain with a Decentralized Oracle Network (DON) using another storage platform such as AWS S3, Google Drive, IPFS, or any other service where the DON can fetch secrets via HTTP.

Read the API multiple calls tutorial before you follow the steps in this example. This tutorial uses the same example, but with a slightly different process:

  1. Instead of relying on the default off-chain option(gists), you will host your encrypted secrets on AWS S3.
  2. Include the encrypted secrets in an offchain-secrets.json file.
  3. Host the secrets file off-chain (AWS S3).
  4. Include the HTTP URL to the file in your Chainlink Functions request.

The functions-build-offchain-secrets task encrypts the secrets and creates the secrets file for you. For reference, you can find the public key for the DON by running the getDONPublicKey function on the Functions Oracle Proxy contract. See the Supported Networks page to find the Functions Oracle Proxy contract for each supported network.

Before you begin

  1. Complete the setup steps in the Getting Started guide: The Getting Started Guide shows you how to set up your environment with the necessary tools for these tutorials. You can re-use the same consumer contract for each of these tutorials.

  2. Make sure to understand the API multiple calls guide.

  3. Make sure your subscription has enough LINK to pay for your requests. Read Get Subscription details to learn how to check your subscription balance. If your subscription runs out of LINK, follow the Fund a Subscription guide.

  4. Check out the tutorials branch of the Chainlink Functions Starter Kit. You can locate this tutorial in the /tutorials/7-offchain-secrets directory.

    git checkout tutorials

  5. Get a free API key from CoinMarketCap.

  6. Run npx env-enc set to add an encrypted COINMARKETCAP_API_KEY to your .env.enc file.

    npx env-enc set

  7. Prepare the store for your encrypted secrets file.

    1. Create a AWS free tier account.
    2. Follow these steps to create a AWS S3 bucket. Choose a name for your bucket, set ACLs enabled, and turn off Block all public access.

Tutorial

This tutorial is configured to get the median BTC/USD price from multiple data sources. For a detailed explanation of the code example, read the Explanation section.

  • Open config.js. Note the args value is ["1", "bitcoin", "btc-bitcoin"]. These arguments are BTC IDs at CoinMarketCap, CoinGecko, and Coinpaprika. You can adapt args to fetch other asset prices. See the API docs for CoinMarketCap, CoinGecko, and CoinPaprika for details. For more information about the request, read the request config section.
  • Open source.js to analyze the JavaScript source code. Read the source code explanation for a more detailed explanation of the request source file.

Build Off-chain Secrets

Before you make a request, prepare the secrets file and host it off-chain:

  1. Encrypt the secrets with the public key of the DON and store them in the offchain-secrets.json file. The --network flag is required because each network has a unique DON with a different public key.

    npx hardhat functions-build-offchain-secrets --network REPLACE_NETWORK --configpath REPLACE_CONFIG_PATH

    Example:

    $ npx hardhat functions-build-offchain-secrets --network polygonMumbai --configpath tutorials/7-offchain-secrets/config.js
secp256k1 unavailable, reverting to browser version
Using public keys from FunctionsOracle contract 0xeA6721aC65BCeD841B8ec3fc5fEdeA6141a0aDE4 on network mumbai

Wrote offchain secrets file to offchain-secrets.json

  2. Follow these steps to upload the file offchain-secrets.json to your AWS S3 bucket.

  3. To make the file publically accessible without authentication:

    1. Find the file in the bucket list, and click on it to open the object overview.
    2. Click on the Permissions tab to display the Access control list (ACL).
    3. Click on Edit.
    4. Set Everyone (public access) Objects read, then confirm. This action makes the object readable by anyone on the internet.
    5. Note the object URL.
    6. To verify that the URL is publicly readable without authentication, open a new browser tab and copy/paste the object URL in the browser location bar. After you hit Enter , the browser will display the content of your encrypted secrets file.

  4. Open config.js. Fill in the secretsURLs variable. For example: secretsURLs: ["https://clfunctions.s3.eu-north-1.amazonaws.com/offchain-secrets.json"]. Note: When you make requests, any URLs in secretsURL are encrypted so no third party can view them.

Simulation

The Chainlink Functions Hardhat Starter Kit includes a simulator to test your Functions code on your local machine. The functions-simulate command executes your code in a local runtime environment and simulates an end-to-end fulfillment. This helps you to fix issues before you submit functions to the Decentralized Oracle Network.

Run the functions-simulate task to run the source code locally and make sure config.js and source.js are correctly written:

npx hardhat functions-simulate --configpath REPLACE_CONFIG_PATH

Example:

$ npx hardhat functions-simulate --configpath tutorials/7-offchain-secrets/config.js
secp256k1 unavailable, reverting to browser version

__Compiling Contracts__
Nothing to compile
Duplicate definition of Transfer (Transfer(address,address,uint256,bytes), Transfer(address,address,uint256))

Executing JavaScript request source code locally...

__Console log messages from sandboxed code__
Median Bitcoin price: $28148.36

__Output from sandboxed source code__
Output represented as a hex string: 0x00000000000000000000000000000000000000000000000000000000002af374
Decoded as a uint256: 2814836

__Simulated On-Chain Response__
Response returned to client contract represented as a hex string: 0x00000000000000000000000000000000000000000000000000000000002af374
Decoded as a uint256: 2814836

Gas used by sendRequest: 411773
Gas used by client callback function: 75029

Reading the output of the example above, you can note that the BTC/USD median price is: 28148.36 USD. Because Solidity does not support decimals, we move the decimal point so that the value looks like the integer 2814836 before returning the bytes encoded value 0x00000000000000000000000000000000000000000000000000000000002af374 in the callback. Read the source code explanation for a more detailed explanation.

Request

Send a request to the Decentralized Oracle Network to fetch the asset price. Run the functions-request task with the subid (subscription ID) and contract parameters. This task passes the functions JavaScript source code and any arguments and secrets when calling the executeRequest function in your deployed FunctionsConsumer contract. Read the functionsConsumer section for a more detailed explanation about the consumer contract.

npx hardhat functions-request --subid REPLACE_SUBSCRIPTION_ID --contract REPLACE_CONSUMER_CONTRACT_ADDRESS --network REPLACE_NETWORK --configpath REPLACE_CONFIG_PATH

Example (You will see several compile warnings, but no errors):

$ npx hardhat functions-request --subid 443 --contract 0x4B4BA2Fd6b93aDF8d6b6002E10540E58394388Ea --network polygonMumbai --configpath tutorials/7-offchain-secrets/config.js
secp256k1 unavailable, reverting to browser version
Estimating cost if the current gas price remains the same...

The transaction to initiate this request will charge the wallet (0x9d087fC03ae39b088326b67fA3C788236645b717):
0.000561571505990096 MATIC, which (using mainnet value) is $0.0006155385277157443

If the request's callback uses all 100,000 gas, this request will charge the subscription:
0.200148071812467286 LINK

Continue? Enter (y) Yes / (n) No
y
Simulating Functions request locally...

__Console log messages from sandboxed code__
Median Bitcoin price: $28148.36

__Output from sandboxed source code__
Output represented as a hex string: 0x00000000000000000000000000000000000000000000000000000000002af374
Decoded as a uint256: 2814836

WARNING: No secrets found for node 0xca46169B34e00cadABB8eCbffA34AE4d1F7050E4.  That node will use default secrets specified by the "0x0" entry.
WARNING: No secrets found for node 0x7a0FD7A68d0257139C9a90C130Fb732E6d997C4B.  That node will use default secrets specified by the "0x0" entry.
WARNING: No secrets found for node 0x4225387E43E066598300E6EF18Af183060B4145b.  That node will use default secrets specified by the "0x0" entry.
WARNING: No secrets found for node 0x42918d83b9298113274420350FD901D9aC382B89.  That node will use default secrets specified by the "0x0" entry.

⣾ Request 0x1d4affdb139f35980da5200db0ef22df1f408ce6e68835b7665dce36e9dec784 has been initiated. Waiting for fulfillment from the Decentralized Oracle Network...


ℹ Transaction confirmed, see https://mumbai.polygonscan.com/tx/0x0d321a8fbe9fb44be1409f08039a0ddaaa114b612afa00a2fc8c17c4367aa81c for more details.

Actual amount billed to subscription #443:
┌──────────────────────┬─────────────────────────────┐
│         Type         │           Amount            │
├──────────────────────┼─────────────────────────────┤
│  Transmission cost:  │  0.000101970932120464 LINK  │
│      Base fee:       │          0.2 LINK           │
│                      │                             │
│     Total cost:      │  0.200101970932120464 LINK  │
└──────────────────────┴─────────────────────────────┘


✔ Request 0x1d4affdb139f35980da5200db0ef22df1f408ce6e68835b7665dce36e9dec784 fulfilled! Data has been written on-chain.

Response returned to client contract represented as a hex string: 0x00000000000000000000000000000000000000000000000000000000002af47c
Decoded as a uint256: 2815100

The output of the example above gives you the following information:

  • The executeRequest function was successfully called in the FunctionsConsumer contract. The transaction in this example is 0x0d321a8fbe9fb44be1409f08039a0ddaaa114b612afa00a2fc8c17c4367aa81c.
  • The request ID is 0x1d4affdb139f35980da5200db0ef22df1f408ce6e68835b7665dce36e9dec784.
  • The DON successfully fulfilled your request. The total cost was: 0.200101970932120464 LINK.
  • The consumer contract received a response in bytes with a value of 0x00000000000000000000000000000000000000000000000000000000002af47c. Decoding the response off-chain to uint256 gives you a result of 2815100.

At any time, you can run the functions-read task with the contract parameter to read the latest received response.

npx hardhat functions-read  --contract REPLACE_CONSUMER_CONTRACT_ADDRESS --network REPLACE_NETWORK --configpath REPLACE_CONFIG_PATH

Example:

$ npx hardhat functions-read  --contract 0x4B4BA2Fd6b93aDF8d6b6002E10540E58394388Ea  --network polygonMumbai --configpath tutorials/7-offchain-secrets/config.js
secp256k1 unavailable, reverting to browser version
Reading data from Functions client contract 0x4B4BA2Fd6b93aDF8d6b6002E10540E58394388Ea on network mumbai

On-chain response represented as a hex string: 0x00000000000000000000000000000000000000000000000000000000002af47c
Decoded as a uint256: 2815100

Explanation

FunctionsConsumer.sol

  • To write a Chainlink Functions consumer contract, your contract must import FunctionsClient.sol. You can read the API reference: FunctionsClient.

    This contract is not available in an NPM package, so you must download and import it from within your project.

    import {Functions, FunctionsClient} from "./dev/functions/FunctionsClient.sol";

  • Use the Functions.sol library to get all the functions needed for building a Chainlink Functions request. You can read the API reference: Functions.

    using Functions for Functions.Request;

  • The latest request id, latest received response, and latest received error (if any) are defined as state variables. Note latestResponse and latestError are encoded as dynamically sized byte array bytes, so you will still need to decode them to read the response or error:

    bytes32 public latestRequestId;
bytes public latestResponse;
bytes public latestError;

  • We define the OCRResponse event that your smart contract will emit during the callback

    event OCRResponse(bytes32 indexed requestId, bytes result, bytes err);

  • Pass the oracle address for your network when you deploy the contract:

    constructor(address oracle) FunctionsClient(oracle)

  • At any time, you can change the oracle address by calling the updateOracleAddress function.

  • The two remaining functions are:

    • executeRequest for sending a request. It receives the JavaScript source code, encrypted secrets, list of arguments to pass to the source code, subscription id, and callback gas limit as parameters. Then:

      • It uses the Functionslibrary to initialize the request and add any passed encrypted secrets or arguments. You can read the API Reference for Initializing a request, adding secrets, and adding arguments.

        Functions.Request memory req;
req.initializeRequest(Functions.Location.Inline, Functions.CodeLanguage.JavaScript, source);
if (secrets.length > 0) {
  req.addRemoteSecrets(secrets);
}
if (args.length > 0) req.addArgs(args);

      • It sends the request to the oracle by calling the FunctionsClient sendRequest function. You can read the API reference for sending a request. Finally, it stores the request id in latestRequestId.

        bytes32 assignedReqID = sendRequest(req, subscriptionId, gasLimit);
latestRequestId = assignedReqID;

    • fulfillRequest to be invoked during the callback. This function is defined in FunctionsClient as virtual (read fulfillRequest API reference). So, your smart contract must override the function to implement the callback. The implementation of the callback is straightforward: the contract stores the latest response and error in latestResponse and latestError before emitting the OCRResponse event.

      latestResponse = response;
latestError = err;
emit OCRResponse(requestId, response, err);

config.js

Read the Request Configuration section for a detailed description of each setting. In this example, the settings are the following:

  • codeLocation: Location.Inline: The JavaScript code is provided within the request.
  • codeLanguage: CodeLanguage.JavaScript: The source code is developed in the JavaScript language.
  • source: fs.readFileSync(path.resolve(__dirname, "source.js")).toString(): The source code must be a script object. This example uses fs.readFileSync to read source.js and calls toString() to get the content as a string object.
  • secrets: { apiKey: process.env.COINMARKETCAP_API_KEY }: JavaScript object which contains secret values. These secrets are encrypted using the DON public key. The process.env.COINMARKETCAP_API_KEY setting means COINMARKETCAP_API_KEY is fetched from the environment variables. Note: secrets is limited to a key-value map that can only contain strings. It cannot include any other types or nested parameters.
  • secretsURLs: ["YOUR_HTTP_URL"]: This is an array that contains the URLs of encrypted secrets.
  • walletPrivateKey: process.env["PRIVATE_KEY"]: This is your EVM account private key. It is used to generate a signature for the encrypted secrets such that an unauthorized third party cannot reuse them.
  • args: ["1", "bitcoin", "btc-bitcoin"]: These arguments are passed to the source code. This example requests the BTC/USD price. These arguments are BTC IDs at CoinMarketCap, CoinGecko, and Coinpaprika. You can adapt args to fetch other asset prices. See the API docs for CoinMarketCap, CoinGecko, and CoinPaprika for details.
  • expectedReturnType: ReturnType.uint256: The response received by the DON is encoded in bytes. Because the asset price is uint256, you must define ReturnType.uint256 to inform users how to decode the response received by the DON.

source.js

To check the expected API responses, run these commands in your terminal:

  • CoinMarketCap:

    curl -X 'GET' \
'https://pro-api.coinmarketcap.com/v1/cryptocurrency/quotes/latest?id=1&convert=USD' \
-H 'accept: application/json' \
-H 'X-CMC_PRO_API_KEY: REPLACE_WITH_YOUR_API_KEY'

  • CoinGecko:

    curl -X 'GET' \
'https://api.coingecko.com/api/v3/simple/price?vs_currencies=USD&ids=bitcoin' \
-H 'accept: application/json'

  • Coinpaprika:

    curl -X 'GET' \
'https://api.coinpaprika.com/v1/tickers/btc-bitcoin' \
-H 'accept: application/json'

The price is located at:

  • CoinMarketCap: data,1,quote,USD,price
  • CoinGecko: bitcoin,usd
  • Coinpaprika: quotes,USD,price

Read the JavaScript code section for a detailed explanation of how to write a compatible JavaScript source code. This JavaScript source code uses Functions.makeHttpRequest to make HTTP requests.

The code is self-explanatory and has comments to help you understand all the steps. The main steps are:

  • Construct the HTTP objects coinMarketCapRequest, coinGeckoRequest, and coinPaprikaRequest using Functions.makeHttpRequest. The values for coinMarketCapCoinId, coinGeckoCoinId, and coinPaprikaCoinId are fetched from the args. See the request config section for details.
  • Make the HTTP calls.
  • Read the asset price from each response.
  • Calculate the median of all the prices.
  • Return the result as a buffer using the Functions.encodeUint256 helper function. Because solidity doesn't support decimals, multiply the result by 100 and round the result to the nearest integer. Note: Read this article if you are new to Javascript Buffers and want to understand why they are important.

Stay updated on the latest Chainlink news