Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Training and testing a single model.

It is possible to run machine learning through Jupyter Notebook, but I would not recommend it. It takes a long time, and it is generally better to run it in the background, from the command line. here, I will outline the process as I have used it.

Step 1. Generate a dataset This is discussed in detail in Sample Datasets.

A typical command looks like this:

time python -m CosmoSim --toml dataset.toml --rnd \
         --csvfile dataset.csv --outfile roulette.csv  \
         --directory images

This is the sample data dataset configuration: dataset.toml.

Step 2. Prepare the dataset for machine learning.

For machine learning, we need separate datasets for training, validation, and testing, which are split from the base dataset. We may also need roulette simulations to use a ground truth for validation. We generate these datasets with the droulette package.

python -m droulette.split problem.toml

The sample file is problem.toml. We can have a look at it:

import json, tomllib as tl
with open( "problem.toml", 'rb') as f:
            toml = tl.load(f)
print( json.dumps( toml, indent=4 ) )
{
    "problem": {
        "filename": "roulette.csv",
        "dataset": "dataset.toml",
        "variables": [
            "sigma",
            "sigma2",
            "theta",
            "xiX",
            "xiY"
        ],
        "nterms": 4
    },
    "dataset": {
        "roulette": "resimulation",
        "training": {
            "filename": "training.csv",
            "size": 12000
        },
        "validation": {
            "filename": "validation.csv",
            "size": 4000
        },
        "testing": {
            "filename": "testing.csv",
            "size": 4000
        }
    }
}

Observe that file and directory names are specified in the config file.

Step 3. Train models. Finally we can train the models.

The default format trains the model in a subdirectory, to allow multiple models of the same data. Here we assume a subdirectory experiment001, containing a configuration file ml.toml. The file may look like this:

import tomllib as tl
with open( "experiment001/ml.toml", 'rb') as f:
            ml = tl.load(f)
print( json.dumps( ml, indent=4 ) )
{
    "settings": {
        "device": "cuda",
        "problem": "problem.toml",
        "model": "regnet_y_32gf"
    },
    "hyperparameters": {
        "learning-rate": 0.001,
        "batch-size": 32,
        "weight-decay": 0.0001
    },
    "training": {
        "num-epochs": 50,
        "patience": 10
    },
    "dataset": {
        "directory": "images",
        "training": {
            "filename": "training.csv"
        },
        "testing": {
            "filename": "testing.csv"
        },
        "validation": {
            "filename": "validation.csv"
        }
    },
    "output": {
        "directory": "experiment001"
    }
}

To train the model, we run the command:

python -m droulette.model --config experiment001/ml.toml

All the results will be placed in this subdirectory. Evaluation of these results is discussed further in Evaluation on the test set.

Lens models
Sample Datasets
Lens models
Evaluation on the test set