44 lines
975 B
Markdown
44 lines
975 B
Markdown
|
# Running Notebooks in Background
|
||
|
|
|
||
|
|
## Option 1: Papermill (Recommended)
|
||
|
|
|
||
|
|
Runs notebooks directly without conversion.
|
||
|
|
|
||
|
|
```bash
|
||
|
|
pip install papermill
|
||
|
|
nohup papermill <notebook>.ipynb output.ipynb > papermill.log 2>&1 &
|
||
|
|
```
|
||
|
|
|
||
|
|
Monitor:
|
||
|
|
```bash
|
||
|
|
tail -f papermill.log
|
||
|
|
```
|
||
|
|
|
||
|
|
## Option 2: Convert to Python Script
|
||
|
|
|
||
|
|
```bash
|
||
|
|
jupyter nbconvert --to script <notebook>.ipynb
|
||
|
|
nohup python <notebook>.py > output.log 2>&1 &
|
||
|
|
```
|
||
|
|
|
||
|
|
**Note:** `%pip install` magic commands need manual removal before running as `.py`
|
||
|
|
|
||
|
|
## Important Notes
|
||
|
|
|
||
|
|
- Ray Tune notebooks require the OCR service running first (Docker)
|
||
|
|
- For Ray workers, imports must be inside trainable functions
|
||
|
|
|
||
|
|
## Example: Ray Tune PaddleOCR
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# 1. Start OCR service
|
||
|
|
cd src/paddle_ocr && docker compose up -d ocr-cpu
|
||
|
|
|
||
|
|
# 2. Run notebook with papermill
|
||
|
|
cd src
|
||
|
|
nohup papermill paddle_ocr_raytune_rest.ipynb output_raytune.ipynb > papermill.log 2>&1 &
|
||
|
|
|
||
|
|
# 3. Monitor
|
||
|
|
tail -f papermill.log
|
||
|
|
```
|