Added supervision library support, batch processing, optimised ONNX with io_binding and memory managment

.gitignore

@@ -1,138 +1,9 @@
-# Byte-compiled / optimized / DLL files
+_old
+*.pyc
 __pycache__/
-*.py[cod]
-*$py.class
+deim_outputs/under/
+deim_outputs/sides/
+*.ipynb
+*.png
+weight/
 
-# C extensions
-*.so
-
-# Distribution / packaging
-.Python
-build/
-develop-eggs/
-dist/
-downloads/
-eggs/
-.eggs/
-lib/
-lib64/
-parts/
-sdist/
-var/
-wheels/
-pip-wheel-metadata/
-share/python-wheels/
-*.egg-info/
-.installed.cfg
-*.egg
-MANIFEST
-
-# PyInstaller
-#  Usually these files are written by a python script from a template
-#  before PyInstaller builds the exe, so as to inject date/other infos into it.
-*.manifest
-*.spec
-
-# Installer logs
-pip-log.txt
-pip-delete-this-directory.txt
-
-# Unit test / coverage reports
-htmlcov/
-.tox/
-.nox/
-.coverage
-.coverage.*
-.cache
-nosetests.xml
-coverage.xml
-*.cover
-*.py,cover
-.hypothesis/
-.pytest_cache/
-
-# Translations
-*.mo
-*.pot
-
-# Django stuff:
-local_settings.py
-db.sqlite3
-db.sqlite3-journal
-
-# Flask stuff:
-instance/
-.webassets-cache
-
-# Scrapy stuff:
-.scrapy
-
-# Sphinx documentation
-docs/_build/
-
-# PyBuilder
-target/
-
-# Jupyter Notebook
-.ipynb_checkpoints
-
-# IPython
-profile_default/
-ipython_config.py
-
-# pyenv
-.python-version
-
-# pipenv
-#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
-#   However, in case of collaboration, if having platform-specific dependencies or dependencies
-#   having no cross-platform support, pipenv may install dependencies that don't work, or not
-#   install all needed dependencies.
-#Pipfile.lock
-
-# PEP 582; used by e.g. github.com/David-OConnor/pyflow
-__pypackages__/
-
-# Celery stuff
-celerybeat-schedule
-celerybeat.pid
-
-# SageMath parsed files
-*.sage.py
-
-# Environments
-.env
-.venv
-env/
-venv/
-ENV/
-env.bak/
-venv.bak/
-
-# Spyder project settings
-.spyderproject
-.spyproject
-
-# Rope project settings
-.ropeproject
-
-# mkdocs documentation
-/site
-
-# mypy
-.mypy_cache/
-.dmypy.json
-dmypy.json
-
-# Pyre type checker
-.pyre/
-
-# PyCharm
-.idea
-
-*.pt
-*.pth
-*.onnx
-*.zip
-*.html
-.DS_Store

.vscode/launch.json

@@ -0,0 +1,65 @@
+{
+    "version": "0.2.0",
+    "configurations": [
+        {
+            "name": "Debug train.py (nproc=1 Simulation)",
+            "type": "python", // Use the Python debugger extension
+            "request": "launch",
+            "program": "${workspaceFolder}/train.py", // Path to your script
+            "console": "integratedTerminal", // Or "internalConsole"
+            "args": [ // Arguments normally passed after train.py
+                "-c",
+                "./deim_hgnetv2_n_coco.yml", // Replace ${model} or use specific file
+                "--use-amp",
+                "--seed=0",
+                "-t",
+                "./deim_dfine_hgnetv2_n_coco_160e.pth"
+                // Add any other args you need
+            ],
+            "env": { // Environment variables torchrun would set for nproc=1
+                "MASTER_ADDR": "localhost",
+                "MASTER_PORT": "7778", // Match your torchrun port or choose another free one
+                "RANK": "0",
+                "WORLD_SIZE": "1",
+                "LOCAL_RANK": "0"
+                // Add PYTHONPATH if your imports require it and VS Code doesn't pick it up automatically
+                // "PYTHONPATH": "${workspaceFolder}/../:${env:PYTHONPATH}" // Example if train.py needs parent dir
+            },
+            "justMyCode": true // Set to false to step into library code
+        },
+        {
+            "name": "Python: Debug Script",
+            "type": "python",
+            "request": "launch",
+            "program": "${workspaceFolder}/tools/inference/torch_inf_super.py",
+            "args": [
+                "-c",
+                "deim_hgnetv2_n_coco.yml",
+                "-r",
+                "deim_outputs/deim_hgnetv2_n_coco/best_stg1_converted.pth",
+                "--input",
+                "66d8c56ea502fd4f902c330e_TC01_cupy.mp4",
+                "--device",
+                "cpu"
+            ],
+            "console": "integratedTerminal",
+            "justMyCode": false
+        },
+        {
+            "name": "Python: train.py",
+            "type": "python",
+            "request": "launch",
+            "program": "train.py",
+            "console": "integratedTerminal",
+            "args": [
+                "-c",
+                "deim_model_under.yml",
+                "--use-amp",
+                "--seed=0",
+                "-t",
+                "base_model.pth"
+            ],
+            "cwd": "${workspaceFolder}"
+        }
+    ]
+}

CLAUDE.md

@@ -0,0 +1,26 @@
+**Prompt:**
+Design a script that facilitates the creation of a new AI training module. The script should perform the following tasks:
+
+1. **Directory Structure**: 
+   - Create a clean directory structure for the new project, ensuring it includes all necessary folders for modules, data, and scripts.
+   - Copy required files and folders from the `_old` directory to the new project directory without altering the logic for training. do not edit any files in _old and do not import from old ever!
+
+2. **Module Creation**:
+   - In the root of the new project directory, create a new Python script that serves as the entry point for both training and inference.
+   - Ensure the new module can be imported easily and allows for training and inference to be executed in fewer than 10 lines of code.
+
+3. **Dependencies**:
+   - Utilize the `supervision` package for annotation purposes, ensuring users can easily add annotated files for training.
+   - Include instructions within the script for users to run the training and inference processes.
+
+4. **Environment Configuration**:
+   - Specify the use of the Python interpreter located at `/home/hidara/miniconda3/envs/deim/bin/python` throughout the project.
+   - Implement linting with `ruff` and type checking with `pyright` to ensure code quality.
+
+5. **User Guidance**:
+   - Add comments and documentation within the script to guide new users who may be unfamiliar with AI training processes, focusing on ease of use for adding new annotated files for training and transfer learning.
+
+6. **Execution**:
+   - Ensure that the script does not run any training code automatically due to the long-running nature of the training sessions on the GPU.
+
+By following these specifications, the resulting script will create a user-friendly environment for new users to engage with AI training and inference, making the process straightforward and accessible. 

DEIM.code-workspace

@@ -0,0 +1,11 @@
+{
+	"folders": [
+		{
+			"path": "."
+		},
+		{
+			"path": "../../datasets"
+		}
+	],
+	"settings": {}
+}