├── .gitignore
├── LICENSE.md
├── README.md
├── images
└── lensai.svg
├── notebooks
├── Example_1
│ └── Singlet_RF_Model_RMS_Spot_Size.ipynb
├── Example_2
│ └── Ray_Path_Failure_Classification_Model.ipynb
├── Example_3
│ └── Double_Gauss_Surrogate_Model.ipynb
├── Example_4
│ └── RL_aspheric_singlet.ipynb
├── Example_5
│ └── SR_GAN_for_wavefront_data.ipynb
└── Example_6
│ ├── Build_Dataset.ipynb
│ └── Misalignment_Prediction_Cooke_Triplet.ipynb
└── requirements.txt
/.gitignore:
--------------------------------------------------------------------------------
1 | **/scripts/
2 | .venv/
3 | **/zmx_files/
4 | **/data/
--------------------------------------------------------------------------------
/LICENSE.md:
--------------------------------------------------------------------------------
1 | MIT License
2 |
3 | Copyright (c) 2024 Kramer Harrison
4 |
5 | Permission is hereby granted, free of charge, to any person obtaining a copy
6 | of this software and associated documentation files (the "Software"), to deal
7 | in the Software without restriction, including without limitation the rights
8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 |
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 |
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 |
6 |
7 | ## Integrated Machine and Deep Learning for Optical Systems
8 |
9 | Welcome to **LensAI**, a collection of studies where machine learning (ML) and deep learning (DL) techniques are applied to the field of optics and optical design. These studies leverage the power of AI to investigate and solve challenges in lens design, wavefront correction, ray tracing, and more, all using the open-source lens design tool [Optiland](https://github.com/HarrisonKramer/optiland) alongside standard scientific libraries.
10 |
11 | > [!NOTE]
12 | > This project is in active development. New notebooks are added regularly.
13 |
14 | # Overview
15 |
16 | This repository contains a series of Jupyter notebooks exploring how ML and DL can be used to enhance and automate various tasks in optical engineering. Topics of interest include:
17 |
18 | - Lens Design Optimization
19 | - Aberration Classification and Correction
20 | - Wavefront Error Prediction and Correction
21 | - Inverse Design of Lens Systems
22 | - Reinforcement Learning for Optical Design
23 | - Generative Lens Design
24 | - Super-resolution Applications in Wavefront Detection
25 | - Deep Learning for Optimal Tolerance Sensitivities
26 | - Surrogate Models for Raytracing
27 |
28 | By integrating AI methodologies into optics, we aim to improve the efficiency and innovation in optical system design, from faster optimizations to generating new designs.
29 |
30 | ## Notebook Summaries
31 |
32 | The following is an overview of existing, in progress, and planned studies:
33 |
34 | - [**Random Forest Regressor to Predict Optimal Lens Properties**](https://github.com/HarrisonKramer/LensAI/blob/main/notebooks/Example_1/Singlet_RF_Model_RMS_Spot_Size.ipynb)
35 | - Demonstrates how to build and train a random forest regressor to predict the radius of curvature of a plano-convex lens in order to minimize the RMS spot size.
36 | - [**Ray Path Failure Classification Model**](https://github.com/HarrisonKramer/LensAI/blob/main/notebooks/Example_2/Ray_Path_Failure_Classification_Model.ipynb)
37 | - Uses logistic regression to predict ray path failures in a Cooke triplet design.
38 | - [**Surrogate Ray Tracing Model Using Neural Networks**](https://github.com/HarrisonKramer/LensAI/blob/main/notebooks/Example_3/Double_Gauss_Surrogate_Model.ipynb)
39 | - Builds a neural network surrogate ray tracing model to increase effective "ray tracing" speed by 10,000x.
40 | - [**Super-Resolution Generative Adversarial Network to Enhance Wavefront Map Data**](https://github.com/HarrisonKramer/LensAI/blob/main/notebooks/Example_5/SR_GAN_for_wavefront_data.ipynb)
41 | - Utilizes a super-resolution GAN (SRGAN) to upscale low-resolution wavefront data into high-resolution data.
42 | - [**Optimization of Aspheric Lenses via Reinforcement Learning**](https://github.com/HarrisonKramer/LensAI/blob/main/notebooks/Example_4/RL_aspheric_singlet.ipynb)
43 | - Reinforcement learning is applied to the optimization of aspheric singlet lenses to generate new lens designs.
44 | - [**Regression Models for Lens Misalignment Prediction Using Wavefront Map Data**](https://github.com/HarrisonKramer/LensAI/blob/main/notebooks/Example_6/Misalignment_Prediction_Cooke_Triplet.ipynb)
45 | - Investigate various regression models to predict lens misalignment in a Cooke triplet design.
46 | - **Reinforcement Learning for Generating New Lens Starting Point Designs** - planned
47 | - **Diffusion Models for Generative Lens Design** - planned
48 | - **Neural Network for Prediction of Lens Seidel Aberration Coefficients from Surface Data** - planned
49 | - **Generative Adversarial Networks for Extrapolating New Lens Starting Point Designs** - planned
50 | - **Regression Model to Extrapolate Aberration Performance Across Spectral Bands** - planned
51 | - **Autoencoders for Wavefront Data Compression** - planned
52 | - **_Insert Your Idea Here_** - if you have an idea for a notebook, consider [opening an issue](https://github.com/HarrisonKramer/LensAI/issues) or [submitting a pull request](https://github.com/HarrisonKramer/LensAI/pulls).
53 |
54 |
55 | # Dependencies
56 |
57 | This repository is built on top of the following dependencies:
58 |
59 | - [optiland](https://github.com/HarrisonKramer/optiland) - For performing optical system simulations and analyses.
60 | - [numpy](https://numpy.org/) - Fundamental package for numerical computation.
61 | - [scipy](https://scipy.org/) - Optimization and scientific functions.
62 | - [scikit-learn](https://scikit-learn.org/stable/index.html) - Machine learning algorithms.
63 | - [torch](https://pytorch.org/) - Deep learning with PyTorch.
64 | - [matplotlib](https://matplotlib.org/) - Plotting and visualization.
65 | - [pandas](https://pandas.pydata.org/) - Data manipulation and analysis.
66 | - [seaborn](https://seaborn.pydata.org/) - Statistical data visualization.
67 | - [tqdm](https://tqdm.github.io/) - Progress bars in Jupyter notebooks.
68 | - [imblearn](https://imbalanced-learn.org/stable/) - Resample imbalanced datasets.
69 | - [gymnasium](https://gymnasium.farama.org/) - Standard API for reinforcement learning.
70 | - [stable-baselines3](https://stable-baselines3.readthedocs.io/en/master/) - Reinforcement learning algorithms in PyTorch.
71 |
72 | # Getting Started
73 |
74 | Explore the various examples in the [Notebooks](https://github.com/HarrisonKramer/LensAI/tree/main/notebooks) directory or run the examples yourself by cloning the repository and installing the required dependencies:
75 |
76 | ```sh
77 | git clone https://github.com/HarrisonKramer/LensAI.git
78 | cd LensAI
79 | pip install -r requirements.txt
80 | ```
81 |
82 | Navigate to a Jupyter notebook file and open in your favorite editor (e.g., [Visual Studio Code](https://code.visualstudio.com/)).
83 |
84 | Note that if you wish to use PyTorch with cuda support, please install torch by following the steps on the [PyTorch website](https://pytorch.org/get-started/locally/).
85 |
86 | # Contributing
87 |
88 | Contributions are welcome! If you have new ideas for optical studies using machine learning or deep learning, feel free to [open an issue](https://github.com/HarrisonKramer/LensAI/issues) or [submit a pull request](https://github.com/HarrisonKramer/LensAI/pulls).
89 |
90 | # References
91 | 1. Yow, A.P., Wong, D., Zhang, Y. et al. Artificial intelligence in optical lens design. Artif Intell Rev 57, 193 (2024).
92 | 2. Tong Yang, Dewen Cheng, and Yongtian Wang, "Designing freeform imaging systems based on reinforcement learning," Opt. Express 28, 30309-30323 (2020).
93 | 3. Bentley, J., Olson, C., & Society of Photo-Optical Instrumentation Engineers. (2012). Field guide to lens design. SPIE
94 | 4. Greivenkamp, J. E., & Society of Photo-Optical Instrumentation Engineers. (2004). Field guide to geometrical optics. SPIE
95 | 5. Côté G, Lalonde J-F, Thibault S (2019a) Extrapolating from lens design databases using deep learning. Opt Express 27(20):28279–28292
96 | 6. Côté G, Lalonde J-F, Thibault S (2021) Deep learning-enabled framework for automatic lens design starting point generation. Opt Express 29(3):3841–3854
97 | 7. Côté G, Zhang Y, Menke C, Lalonde J-F, Thibault S (2022) Inferring the solution space of microscope objective lenses using deep learning. Opt Express 30(5):6531–6545
98 | 8. Dilworth D (1987) Applications of Artificial Intelligence To computer-aided Lens Design. SPIE
99 |
100 | # Contact
101 |
102 | If you have any remarks, requests, issues, suggestions, or find a bug/mistake, please feel free to [open an issue](https://github.com/HarrisonKramer/LensAI/issues) in the repository.
103 |
104 | Kramer Harrison - kdanielharrison@gmail.com
105 |
--------------------------------------------------------------------------------
/images/lensai.svg:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 | LensAI LensAI "
68 | ]
69 | },
70 | "metadata": {},
71 | "output_type": "display_data"
72 | }
73 | ],
74 | "source": [
75 | "lens = CookeTriplet()\n",
76 | "lens.draw()"
77 | ]
78 | },
79 | {
80 | "cell_type": "code",
81 | "execution_count": 6,
82 | "metadata": {},
83 | "outputs": [],
84 | "source": [
85 | "np.random.seed(42)\n",
86 | "\n",
87 | "num_samples = 1_000\n",
88 | "\n",
89 | "delta = 0.5\n",
90 | "\n",
91 | "dx1 = np.random.uniform(-delta, delta, num_samples)\n",
92 | "dy1 = np.random.uniform(-delta, delta, num_samples)\n",
93 | "dz1 = np.random.uniform(-delta, delta, num_samples)\n",
94 | "\n",
95 | "dx2 = np.random.uniform(-delta, delta, num_samples)\n",
96 | "dy2 = np.random.uniform(-delta, delta, num_samples)\n",
97 | "dz2 = np.random.uniform(-delta, delta, num_samples)\n",
98 | "\n",
99 | "dx3 = np.random.uniform(-delta, delta, num_samples)\n",
100 | "dy3 = np.random.uniform(-delta, delta, num_samples)\n",
101 | "dz3 = np.random.uniform(-delta, delta, num_samples)\n",
102 | "\n",
103 | "df = pd.DataFrame({'dx1': dx1, 'dy1': dy1, 'dz1': dz1,\n",
104 | " 'dx2': dx2, 'dy2': dy2, 'dz2': dz2,\n",
105 | " 'dx3': dx3, 'dy3': dy3, 'dz3': dz3})"
106 | ]
107 | },
108 | {
109 | "cell_type": "code",
110 | "execution_count": 7,
111 | "metadata": {},
112 | "outputs": [
113 | {
114 | "name": "stderr",
115 | "output_type": "stream",
116 | "text": [
117 | "100%|██████████| 1000/1000 [18:36<00:00, 1.12s/it]\n"
118 | ]
119 | }
120 | ],
121 | "source": [
122 | "data = np.zeros((num_samples, 3, 36), dtype=np.float32)\n",
123 | "\n",
124 | "for k, row in tqdm(df.iterrows(), total=num_samples):\n",
125 | " lens.reset()\n",
126 | " lens.decenter(0, row['dx1'], row['dy1'], row['dz1'])\n",
127 | " lens.decenter(1, row['dx2'], row['dy2'], row['dz2'])\n",
128 | " lens.decenter(2, row['dx3'], row['dy3'], row['dz3'])\n",
129 | "\n",
130 | " z1 = wavefront.ZernikeOPD(lens, field=(0, 0.0), wavelength=0.55, zernike_type='standard', num_terms=36)\n",
131 | " z2 = wavefront.ZernikeOPD(lens, field=(0, 0.7), wavelength=0.55, zernike_type='standard', num_terms=36)\n",
132 | " z3 = wavefront.ZernikeOPD(lens, field=(0, 1.0), wavelength=0.55, zernike_type='standard', num_terms=36)\n",
133 | "\n",
134 | " data[k, 0] = z1.coeffs\n",
135 | " data[k, 1] = z2.coeffs\n",
136 | " data[k, 2] = z3.coeffs"
137 | ]
138 | },
139 | {
140 | "cell_type": "code",
141 | "execution_count": 8,
142 | "metadata": {},
143 | "outputs": [],
144 | "source": [
145 | "np.save('Cooke_triplet_wavefront_data.npy', data)"
146 | ]
147 | },
148 | {
149 | "cell_type": "code",
150 | "execution_count": 9,
151 | "metadata": {},
152 | "outputs": [],
153 | "source": [
154 | "df.to_csv('Cooke_wavefront_data.csv', index=False)"
155 | ]
156 | }
157 | ],
158 | "metadata": {
159 | "kernelspec": {
160 | "display_name": ".venv",
161 | "language": "python",
162 | "name": "python3"
163 | },
164 | "language_info": {
165 | "codemirror_mode": {
166 | "name": "ipython",
167 | "version": 3
168 | },
169 | "file_extension": ".py",
170 | "mimetype": "text/x-python",
171 | "name": "python",
172 | "nbconvert_exporter": "python",
173 | "pygments_lexer": "ipython3",
174 | "version": "3.12.4"
175 | }
176 | },
177 | "nbformat": 4,
178 | "nbformat_minor": 2
179 | }
180 |
--------------------------------------------------------------------------------
/requirements.txt:
--------------------------------------------------------------------------------
1 | optiland
2 | numpy
3 | pandas
4 | matplotlib
5 | scikit-learn
6 | seaborn
7 | tqdm
8 | torch
9 | imbalanced-learn
10 | gymnasium
11 | stable-baselines3
--------------------------------------------------------------------------------
4 |
5 |