Migrate Data from Neptune¶
This guide shows how to load exported data from Neptune into Comet using the Neptune to Comet exporter script.
The exporter script is available in the neptune-to-comet-exporter repository.
Get the code¶
Clone the repository:
git clone https://github.com/comet-ml/neptune-to-comet-exporter.git
cd neptune-to-comet-exporter
Before you start¶
Get your Neptune credentials:
Neptune project name. You can set it with the environment variable
NEPTUNE_PROJECTor in the command line with--neptune-project-name.Neptune API token. You can get it from Neptune Settings. You can set it with the environment variable
NEPTUNE_API_TOKENor in the command line with--neptune-api-token.
Get your Comet credentials:
Comet workspace. You can set it with the environment variable
COMET_WORKSPACEor in the command line with--comet-workspace.Comet project name. You can set it with the environment variable
COMET_PROJECT_NAMEor in the command line with--comet-project-name.Comet API key. You can get it from Comet Settings. You can set it with the environment variable
COMET_API_KEY.
Install dependencies¶
Install the required Python packages from the requirements.txt file:
pip install -r requirements.txt
Or install them individually:
pip install comet_ml neptune
If you're using a virtual environment (recommended):
python3 -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
pip install -r requirements.txt
Load the data into Comet¶
The following command loads all non-deleted runs from a Neptune project into Comet. Make sure you're in the neptune-to-comet-exporter directory:
python3 neptune_exporter.py \
--neptune-project-name "workspace/project-name" \
--neptune-api-token "$NEPTUNE_API_TOKEN" \
--comet-workspace "your-workspace" \
--comet-project-name "your-project-name"
Make sure to set your Comet API key as an environment variable:
export COMET_API_KEY="your-comet-api-key"
Example¶
export COMET_API_KEY="your-comet-api-key"
python3 neptune_exporter.py \
--neptune-project-name "workspace/project-name" \
--neptune-api-token "your-neptune-api-token" \
--comet-workspace "your-workspace" \
--comet-project-name "your-project-name"
Using environment variables¶
You can also use environment variables instead of command-line arguments:
export NEPTUNE_PROJECT="workspace/project-name"
export NEPTUNE_API_TOKEN="your-neptune-api-token"
export COMET_WORKSPACE="your-workspace"
export COMET_PROJECT_NAME="your-project-name"
export COMET_API_KEY="your-comet-api-key"
python3 neptune_exporter.py \
--neptune-project-name "$NEPTUNE_PROJECT" \
--comet-workspace "$COMET_WORKSPACE" \
--comet-project-name "$COMET_PROJECT_NAME"
What gets exported¶
The exporter transfers the following data from Neptune to Comet:
- Hyperparameters: All parameters from
parametersorhyperparametersnamespace - Metrics: Training and validation metrics
- Final metrics: Metrics from the
finalnamespace - Metadata: Additional metadata from the
metadatanamespace - Owner information: User information from
sys/owner
Verify the migration¶
After running the exporter, check the output for experiment URLs. Each exported run will display a Comet experiment URL:
created comet experiment: https://www.comet.com/your-workspace/your-project-name/experiment-id
Visit these URLs to verify that all data was transferred correctly.
Troubleshooting¶
Missing dependencies¶
If you get ModuleNotFoundError, make sure you've installed the required packages:
pip install comet_ml neptune
Invalid API token¶
If you get a NeptuneInvalidApiTokenException, verify your Neptune API token is correct. You can get it from Neptune Settings.
Project not found¶
Make sure the Neptune project name is in the format workspace/project-name and that you have access to it.
No data transferred¶
If experiments are created but no data appears, check: - The Neptune runs contain data in the expected paths (parameters, train/loss, etc.) - Your Comet API key has write permissions to the workspace - The script completed without errors
Repository¶
The exporter script is maintained in the neptune-to-comet-exporter repository on GitHub. For issues, feature requests, or contributions, please visit the repository.