In this blog post series we will be aiming to replicate the powerful Code Interpreter tool known from ChatGPT. This tool enables dynamic code execution, providing value in scenarios like data analysis, automation and custom workflows.

In this first part we will focus on the technical aspect of running generated code securely in an isolated environment. This ensures that any potentially malicious code cannot harm the surrounding infrastructure.

By the end of this series we will have recreated the core functionality of ChatGPT’s Code Interpreter, which dynamically generates and executes code for end users. Along the way we will explore the technologies, best practices and challenges involved in building such a tool. Stay tuned for a deep dive into both the technical and practical aspects of this topic!

Dynamic Sessions

Recently, the Azure Container Apps Dynamic Sessions became Generally Available . This powerful tool is what allows us to safely execute code in a safe, sandbox environment, as each instance is protected by Hyper-V isolation.

The Dynamic Sessions tool works by creating isolated sessions. Through the API it’s possible to execute custom Python code. Each session has its own file system, which can be accessed to either download or upload files to the session. This allows us to make data available when executing code.

By default the session is prohibited from accessing the internet. This is to prevent code from acting maliciously, such as making denial-of-service attacks. It is possible to enable egress traffic through configuration, but I would advise to be very cautious when doing this. Especially if the code generated is coming from an LLM as we will explore in a follow up post.

The Session Pool has two roles that give other Azure resources access to invoking the API.

  • Container Apps SessionPools Contributor
  • Azure ContainerApps Session Executor

Azure Setup

There are three ways to set up an Azure Container Apps Session Pool - through the Azure Portal, CLI and IaC. We will explore both the CLI and IaC way of doing it.

CLI

As of writing this post you will need the Container Apps extension for the Azure CLI to setup a Session Pool. You can install this with the following CLI command:

1

az extension add --name containerapp --upgrade --allow-preview true -y

With the extension installed you will be able to setup a Session Pool. Use the following command:

1
2
3
4
5
6
7
8

az containerapp sessionpool create \
    --name ca-session-pool \
    --resource-group <RESOURCE_GROUP> \
    --location northeurope \
    --container-type PythonLTS \
    --max-sessions 100 \
    --cooldown-period 300 \
    --network-status EgressDisabled

Bicep

To setup the IaC we will be using Bicep. The following definition will create and configure a Session Pool:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

resource sessionPool 'Microsoft.App/sessionPools@2024-02-02-preview' = {
  name: 'ca-session-pool'
  location: 'northeurope'
  properties: {
    poolManagementType: 'Dynamic'
    containerType: 'PythonLTS'
    scaleConfiguration: {
      maxConcurrentSessions: 100
    }
    dynamicPoolConfiguration: {
      executionType: 'Timed'
      cooldownPeriodInSeconds: 300
    }
    sessionNetworkConfiguration: {
      status: 'EgressDisabled'
    }
  }
}

Notice that we set the network configuration to EgressDisabled. This is what determines if executed code can access the internet. You can set this to EgressEnabled if you want to, but be cautious with what code you then allow to be executed. There may very well be use cases where you want to enabled egress, but in this series where we will be executing dynamically generated code we will keep it disabled.

API Implementation

The Session Pool works by invoking API endpoints that interacts with a session. I will go over the most important endpoints available as of writing this blog post, but consider checking the official documentation , as this is evolving quickly. We will also explore an API implementation in C# to quickly get started using Session Pools.

Primary Endpoints

Code Execution

This endpoint is the primary endpoint - the one that will execute custom Python code.
The endpoint takes a session ID which isolates the code execution.
As of writing this there is no functionality for codeInputType or executionType.

File Upload

This endpoint allows us to upload files to a given session. The files will then be available for the code execution.
The files will be stored in the /mnt/data/ directory.

C# API Wrapper

To ease the integration of Dynamic Sessions, I have created a wrapper for the API that is free to use. The full code implementation is available on GitHub .

Features

  • Code Execution: Execute custom Python code within a session.
  • File Management: Upload and download files to/from a session.

Usage

Below is a quick example of how to use the wrapper to create a session and execute code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

var client = new DynamicSessionsClient(options, httpClientFactory, credentials, logger);

var sessionId = "session-id";

var uploadFileRequest = new UploadFileRequest
{
    Identifier = sessionId,
    FileName = "data.csv",
    FileContent = File.OpenRead("data.csv")
};
await client.UploadFileAsync(uploadFileRequest);

var code = "import pandas as pd\n\n# Load the data\nfile_path = '/mnt/data/data.csv'\ndata = pd.read_csv(file_path)\n\n# Display the data\nprint(data.head())";

var codeExecutionRequest = new CodeExecutionRequest
{
    Identifier = sessionId,
    PythonCode = code
};
var result = await client.ExecuteCodeAsync(codeExecutionRequest);

This implementation allows you to quickly get started using Dynamic Sessions in your C# applications. For more detailed examples and documentation, please refer to the GitHub repository .
If you would like to see this wrapper as a NuGet package, let me know in the comments!

Conclusion

In this first part of the blog series we have laid the foundation for replicating the Code Interpreter by exploring Azure Container Apps Dynamic Sessions. From understanding the basics of secure and isolated code execution to setting up a Session Pool using either CLI or IaC, we have covered the essential steps needed to get started. With the provided C# wrapper for the API we are now ready to integrate these capabilities into applications.

In the next part of this series we will explore how to leverage LLMs like GPT to generate executable code for us, laying the groundwork for fully replicating ChatGPT’s Code Interpreter. Stay tuned and feel free to share your thoughts, questions or experiences in the comments below - I would love to hear from you!