Object Detection EX: Machine Learning Optimization Framework

Overview

Object Detection EX is a comprehensive research framework I developed during my time at Mayachitra to analyze object detection data produced by Detectron, optimize machine learning models through evolutionary algorithms, and conduct systematic experiments on high-dimensional feature spaces. The framework addresses the complex challenges of working with object detection annotations, feature extraction, and hyperparameter optimization in a research environment.

The Challenge

Working with object detection data presents unique challenges: feature spaces are high-dimensional, annotations exist in multiple formats (JSON ground truth labels vs. HDF5 feature files), and finding optimal model architectures requires extensive experimentation. Traditional grid search approaches become computationally prohibitive when dealing with large hyperparameter spaces, especially when working with neural networks that require significant training time.

Key Components

1. Data Mapping and Annotation Processing

The framework bridges the gap between different annotation formats by mapping ground truth labels from JSON files to HDF5 feature files produced by a modified version of Roy Tseng's Detectron implementation. This was built using the CVJSON library, which provides a powerful interface for working with computer vision JSON files.

Key Features:

• Seamless mapping between JSON annotations and HDF5 feature representations
• Support for class grouping and label manipulation
• Integration with CVJSON for efficient JSON handling

2. Experiment Pipelines

Two comprehensive experiment pipelines were developed to handle different types of machine learning models:

• Scikit-learn Pipeline: For traditional classifiers and models
• PyTorch Pipeline: For deep learning models and neural networks

Both pipelines automatically handle experiment execution, data collection, and report generation, streamlining the research workflow significantly.

3. High-Dimensional Data Visualization

Given that object detection data exists in high-dimensional feature spaces, traditional visualization techniques fail to preserve meaningful structure. The framework includes specialized plotting capabilities designed for t-SNE (t-Distributed Stochastic Neighbor Embedding), which is uniquely suited for preserving discrimination in high-dimensional data when reducing to 2D or 3D visualizations.

4. Hyperparameter Optimization

Two optimization strategies were implemented to address the NP-complete nature of hyperparameter search:

Grid Search: Exhaustive search through a predefined hyperparameter space. Useful when you have a good intuition about where optimal parameters lie, but computationally expensive for large search spaces.

Genetic Algorithm (Tournament-Based Selection): An evolutionary optimization technique that approximates optimal solutions through selection, crossover, and mutation. This approach is particularly effective for finding optimal hidden layer sizes and training epochs in neural networks.

Technical Deep Dive: Genetic Algorithm Implementation

Why Genetic Algorithms?

Hyperparameter optimization is fundamentally an NP-complete problem—we know solutions exist, but finding the best one is computationally infeasible for large search spaces. While reinforcement learning has gained prominence in Neural Architecture Search (NAS), genetic algorithms have shown competitive results, with Uber's research demonstrating that GAs can sometimes find better solutions than RL approaches.

How the Genetic Algorithm Works

The GA represents hyperparameters as chromosomes, where each chromosome contains genes (individual parameters like hidden_neurons and epochs). Each gene is encoded as a binary string, allowing for fine-grained control over the search space.

Key Operations:

• Fitness Evaluation: Model performance (accuracy, F1-score, etc.) determines fitness
• Tournament Selection: Competing chromosomes are selected based on fitness
• Crossover: Chromosomes exchange genetic material (parameter values) to explore new combinations
• Mutation: Random modifications prevent convergence to local optima

The algorithm balances exploration (finding new promising regions) with exploitation (refining known good solutions) through carefully tuned mutation and crossover rates.

Practical Implementation

The framework allows researchers to define optimization constraints and pass model class definitions directly to the GA:

optimization_constraints = {
    "epochs": (25, 1000), 
    "hidden_neurons": (10, 100)
}
 
best_params = tournament_GA_pytorch(
    models.logic_net, 
    X_train, y_train, 
    X_test, y_test,
    optimization_constraints=optimization_constraints,
    generations=30, 
    population_size=30
)

Algorithm Caveats and Design Decisions

Through experimentation, I discovered critical trade-offs:

• High Mutation Rates: Too much randomness prevents convergence to optimal solutions. Evolutionary algorithms naturally explore in all directions, so excessive mutation amplifies this exploration at the cost of exploitation.

• Crossover Strategy: The implementation performs crossover across entire chromosomes rather than gene-specific crossover. This means a hidden_neurons gene can swap with portions of an epochs gene, which provides more exploration but requires careful mutation tuning to escape local minima.

• Gene Length: The binary encoding length determines the resolution of parameter values within the optimization constraints. Longer genes provide finer granularity but increase computational complexity.

Technologies and Tools

• Python: Core implementation language
• PyTorch: Deep learning framework for neural network models
• Scikit-learn: Traditional ML classifiers and utilities
• CVJSON: Library for handling computer vision JSON annotations
• Detectron: Modified version for HDF5 feature extraction
• NumPy: Numerical computing and array operations
• t-SNE: High-dimensional data visualization
• HDF5: Efficient storage and retrieval of large feature datasets

Research Applications

The framework was designed for analyzing maritime object detection datasets, specifically focusing on boat classification tasks. Key capabilities included:

• Class Grouping: Combining semantically similar classes (e.g., different variants of the same boat type) for more robust classification
• Feature Analysis: Extracting and analyzing high-dimensional features from object detection models
• Model Optimization: Finding optimal neural network architectures for specific detection tasks
• Comparative Studies: Systematic evaluation of different classifiers and architectures

Challenges and Solutions

Challenge 1: Data Format Mismatch

• Problem: Ground truth annotations in JSON format vs. features in HDF5 format
• Solution: Built mapping layer using CVJSON to align annotations with extracted features

Challenge 2: High-Dimensional Visualization

• Problem: Traditional visualization techniques fail with high-dimensional object detection features
• Solution: Implemented t-SNE-specific plotting pipeline that preserves discrimination in lower dimensions

Challenge 3: Computational Efficiency

• Problem: Grid search becomes intractable for large hyperparameter spaces
• Solution: Genetic algorithm provides efficient approximation of optimal solutions

Challenge 4: Reproducibility

• Problem: Research experiments need systematic tracking and reporting
• Solution: Automated experiment pipelines that collect metrics and generate comprehensive reports

Outcome

Object Detection EX successfully streamlined the research workflow for object detection analysis, reducing experiment setup time from hours to minutes. The genetic algorithm implementation proved particularly valuable, finding optimal hyperparameters more efficiently than exhaustive grid search while maintaining comparable or better results.

The framework demonstrated the power of evolutionary algorithms in machine learning optimization, particularly for neural network hyperparameter tuning. The systematic approach to class grouping, feature analysis, and model evaluation provided a solid foundation for maritime object detection research.

More importantly, the project taught me valuable lessons about balancing exploration and exploitation in optimization algorithms, the importance of domain-specific data handling tools, and the critical role of visualization in understanding high-dimensional machine learning data.