├── .gitignore ├── 2025_BuildingBridges ├── monthly-reports │ ├── pr05_2025_July_report_Stephan_Max.png │ ├── pr05_2025_August_report_Stephan_Max.png │ ├── pr05_2025_August_report_Claire_Peng_closedPRs.png │ ├── pr05_2025_September_report_Claire_Peng_images │ │ ├── 1.png │ │ ├── 2.png │ │ ├── 3.png │ │ ├── 4.png │ │ ├── 5.png │ │ ├── 6.png │ │ ├── 7.png │ │ └── 8.png │ ├── README.md │ ├── pr05_2025_July_report_Stephan_Max.md │ ├── pr05_2025_August_report_Stephan_Max.md │ ├── pr05_2025_September_report_Stephan_Max.md │ ├── pr05_2025_September_report_Vaivaswat_Dubey.md │ ├── pr05_2025_July_report_Claire_Peng.md │ ├── pr05_2025_September_report_Claire_Peng.md │ ├── pr05_2025_August_report_Vaivaswat_Dubey.md │ ├── pr05_2025_July_report_Vaivaswat_Dubey.md │ └── pr05_2025_August_report_Claire_Peng.md └── final-reports │ ├── pr05_2025_final_report_Claire_Peng │ ├── images │ │ ├── hoverJsDocs.gif │ │ └── swagger-microsite.png │ └── README.md │ ├── README.md │ ├── pr05_2025_final_report_Stephan_Max.md │ └── pr05_2025_final_report_Vaivaswat_Dubey.md ├── 2024_NewBeginnings ├── monthly-reports │ ├── README.md │ ├── pr05_2024_July_report_Miaoye_Que.md │ ├── pr05_2024_September_report_Claudine_Chen.md │ ├── pr05_2024_July_report_Dora_Do.md │ ├── pr05_2024_August_report_Dora_Do.md │ ├── pr05_2024_July_report_Nahee_Kim.md │ ├── pr05_2024_August_report_Nahee_Kim.md │ ├── pr05_2024_August_report_Claudine_Chen.md │ ├── pr05_2024_July_report_Claudine_Chen.md │ ├── pr05_2024_July_report_Diya_Solanki.md │ ├── pr05_2024_September_report_Miaoye_Que.md │ ├── pr05_2024_September_report_Dora_Do.md │ ├── pr05_2024_August_report_Miaoye_Que.md │ ├── pr05_2024_August_report_Diya_Solanki.md │ ├── pr05_2024_September_report_Diya_Solanki.md │ └── pr05_2024_September_report_Nahee_Kim.md └── final-reports │ ├── README.md │ ├── pr05_2024_final_report_Miaoye_Que.md │ ├── pr05_2024_Final_report_DoraDo.md │ ├── pr05_2024_final_report_Claudine_Chen.md │ └── pr05_2024_final_report_Diya_Solanki.md ├── README.md └── CODE_OF_CONDUCT.md /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store -------------------------------------------------------------------------------- /2025_BuildingBridges/monthly-reports/pr05_2025_July_report_Stephan_Max.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/processing/pr05-grant/HEAD/2025_BuildingBridges/monthly-reports/pr05_2025_July_report_Stephan_Max.png -------------------------------------------------------------------------------- /2025_BuildingBridges/monthly-reports/pr05_2025_August_report_Stephan_Max.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/processing/pr05-grant/HEAD/2025_BuildingBridges/monthly-reports/pr05_2025_August_report_Stephan_Max.png -------------------------------------------------------------------------------- /2025_BuildingBridges/monthly-reports/pr05_2025_August_report_Claire_Peng_closedPRs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/processing/pr05-grant/HEAD/2025_BuildingBridges/monthly-reports/pr05_2025_August_report_Claire_Peng_closedPRs.png -------------------------------------------------------------------------------- /2025_BuildingBridges/monthly-reports/pr05_2025_September_report_Claire_Peng_images/1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/processing/pr05-grant/HEAD/2025_BuildingBridges/monthly-reports/pr05_2025_September_report_Claire_Peng_images/1.png -------------------------------------------------------------------------------- /2025_BuildingBridges/monthly-reports/pr05_2025_September_report_Claire_Peng_images/2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/processing/pr05-grant/HEAD/2025_BuildingBridges/monthly-reports/pr05_2025_September_report_Claire_Peng_images/2.png -------------------------------------------------------------------------------- /2025_BuildingBridges/monthly-reports/pr05_2025_September_report_Claire_Peng_images/3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/processing/pr05-grant/HEAD/2025_BuildingBridges/monthly-reports/pr05_2025_September_report_Claire_Peng_images/3.png -------------------------------------------------------------------------------- /2025_BuildingBridges/monthly-reports/pr05_2025_September_report_Claire_Peng_images/4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/processing/pr05-grant/HEAD/2025_BuildingBridges/monthly-reports/pr05_2025_September_report_Claire_Peng_images/4.png -------------------------------------------------------------------------------- /2025_BuildingBridges/monthly-reports/pr05_2025_September_report_Claire_Peng_images/5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/processing/pr05-grant/HEAD/2025_BuildingBridges/monthly-reports/pr05_2025_September_report_Claire_Peng_images/5.png -------------------------------------------------------------------------------- /2025_BuildingBridges/monthly-reports/pr05_2025_September_report_Claire_Peng_images/6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/processing/pr05-grant/HEAD/2025_BuildingBridges/monthly-reports/pr05_2025_September_report_Claire_Peng_images/6.png -------------------------------------------------------------------------------- /2025_BuildingBridges/monthly-reports/pr05_2025_September_report_Claire_Peng_images/7.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/processing/pr05-grant/HEAD/2025_BuildingBridges/monthly-reports/pr05_2025_September_report_Claire_Peng_images/7.png -------------------------------------------------------------------------------- /2025_BuildingBridges/monthly-reports/pr05_2025_September_report_Claire_Peng_images/8.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/processing/pr05-grant/HEAD/2025_BuildingBridges/monthly-reports/pr05_2025_September_report_Claire_Peng_images/8.png -------------------------------------------------------------------------------- /2025_BuildingBridges/final-reports/pr05_2025_final_report_Claire_Peng/images/hoverJsDocs.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/processing/pr05-grant/HEAD/2025_BuildingBridges/final-reports/pr05_2025_final_report_Claire_Peng/images/hoverJsDocs.gif -------------------------------------------------------------------------------- /2025_BuildingBridges/final-reports/pr05_2025_final_report_Claire_Peng/images/swagger-microsite.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/processing/pr05-grant/HEAD/2025_BuildingBridges/final-reports/pr05_2025_final_report_Claire_Peng/images/swagger-microsite.png -------------------------------------------------------------------------------- /2024_NewBeginnings/monthly-reports/README.md: -------------------------------------------------------------------------------- 1 | Monthly reports from the 2024 `pr05` cohort will be stored here. 2 | 3 | Please use the following naming convention: `pr05___report__.md`. 4 | 5 | For example, if your name is **Alice Pleasance Liddell** and you are writing your report for **July 2024**, the file should be called: 6 | 7 | `pr05_2024_July_report_Alice_PleasanceLiddell.md` 8 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # `pr05` 2 | 3 | The Processing Foundation Software Development Grant or `pr05` (pronounced “pros”) is a grant and mentorship initiative by the Processing Foundation designed to support the professional growth of early to mid-career software developers through hands-on involvement in open-source projects. Participation in the program is entirely online. 4 | 5 | To see more about current and past program as well as any running open call, see the [`pr05` Wiki](https://github.com/processing/pr05-grant/wiki). 6 | -------------------------------------------------------------------------------- /2025_BuildingBridges/monthly-reports/README.md: -------------------------------------------------------------------------------- 1 | Monthly reports from the 2025 `pr05` cohort will be stored here. 2 | 3 | Please use the following naming convention: `pr05___report__.md`. 4 | 5 | For example, if your name is **Alice Pleasance Liddell** and you are writing your report for **July 2025**, the file should be called: 6 | 7 | `pr05_2025_July_report_Alice_PleasanceLiddell.md` 8 | 9 | ## Report Structure 10 | 11 | The exact structure of the monthly reports is up to you, but it should include the following information: 12 | 13 | - **Overview** — A brief summary of the month’s focus and achievements. 14 | - **Key Decisions and Progress** — Major decisions made, technologies chosen, and progress on features. 15 | - **Challenges and Solutions** — Difficulties encountered and how they were addressed. 16 | - **Next Steps** — Planned work for the upcoming month. 17 | - **Relevant commits/issues/PRs** — Links to key commits, issues, or pull requests. -------------------------------------------------------------------------------- /2024_NewBeginnings/final-reports/README.md: -------------------------------------------------------------------------------- 1 | # Final Reports 2 | 3 | This directory contains comprehensive final reports from the 2024 pr05 cohort projects, documenting the project journey, covering technical decisions, challenges, tasks completed, limitations, and future steps. 4 | 5 | ## Naming Files 6 | 7 | Please use the following naming convention: 8 | 9 | `pr05__final_report__.md`. 10 | 11 | For example, if your name is Katherine Johnson, the file should be called: 12 | 13 | `pr05_2024_final_report_Katherine_Johnson.md` 14 | 15 | ## Report Structure 16 | 17 | Reports should cover the project as a whole, including but not limited to: 18 | 19 | - **Technical decisions** — Key choices made and alternative options considered. 20 | - **Challenges** — Difficulties encountered and how they were addressed. 21 | - **Tasks completed** — Summary of major accomplishments. 22 | - **Limitations** — Constraints or limitations of the current project. 23 | - **Work remaining and next steps** — Suggested future improvements. 24 | - **Relevant commits/PRs** — Links to key commits or pull requests. 25 | 26 | For personal reflections and broader project insights, see our wrap-up posts on [Medium](https://medium.com/processing-foundation). 27 | -------------------------------------------------------------------------------- /2025_BuildingBridges/final-reports/README.md: -------------------------------------------------------------------------------- 1 | # Final Reports 2 | 3 | This directory contains comprehensive final reports from the 2025 pr05 cohort projects, documenting the project journey, covering technical decisions, challenges, tasks completed, limitations, and future steps. 4 | 5 | ## Naming Files 6 | 7 | Please use the following naming convention: 8 | 9 | `pr05__final_report__.md`. 10 | 11 | For example, if your name is Katherine Johnson, the file should be called: 12 | 13 | `pr05_2025_final_report_Katherine_Johnson.md` 14 | 15 | ## Report Structure 16 | 17 | Reports should cover the project as a whole, including but not limited to: 18 | 19 | - **Technical decisions** — Key choices made and alternative options considered. 20 | - **Challenges** — Difficulties encountered and how they were addressed. 21 | - **Tasks completed** — Summary of major accomplishments. 22 | - **Limitations** — Constraints or limitations of the current project. 23 | - **Work remaining and next steps** — Suggested future improvements. 24 | - **Relevant commits/PRs** — Links to key commits or pull requests. 25 | 26 | For personal reflections and broader project insights, see our wrap-up posts on [Medium](https://medium.com/processing-foundation). 27 | -------------------------------------------------------------------------------- /2024_NewBeginnings/monthly-reports/pr05_2024_July_report_Miaoye_Que.md: -------------------------------------------------------------------------------- 1 | # Revamp the Friendly Error System (FES) in p5.js 2 | 3 | ## Completed Tasks 4 | 5 | - Conducted research for the project's 2 top priorities: discover accidentally overridden constants and functions, and parameter validation. 6 | - Based on research, prior work, and guidance from mentors, drafted a [project plan](https://github.com/sproutleaf/pr05-fes/blob/main/project_plan.md) with proposed approaches, road map, and project timeline. 7 | 8 | ## Challenges Encountered 9 | 10 | The month of July was pretty smooth for me, because my focus was on research, experimentation, and writing. I anticipate encountering many more challenges once the implementation stage begins. 11 | 12 | ## Plan For the Upcoming Month 13 | 14 | I am currently on a break between grad school and a full-time job that is going to start in early September. Due to the way my obligations are set up, August will be a big implementation month for me with regards to grant work. On a high level, I plan to finish, or be close to finishing, the implementation of both FES priorities (accidentally overridden constants and functions, parameter validation). For more detailed plans, please refer to the [road map section](https://github.com/sproutleaf/pr05-fes/blob/main/project_plan.md) of my project plan. 15 | -------------------------------------------------------------------------------- /2024_NewBeginnings/monthly-reports/pr05_2024_September_report_Claudine_Chen.md: -------------------------------------------------------------------------------- 1 | # August Monthly Report - for Simplifying the Workflow for Libraries, Tools, and Modes 2 | ## Claudine Chen 3 | 4 | ### Work Accomplished 5 | The Processing library template was completed to the point that we could invite feedback. 6 | This was done by announcement in the newsletter. 7 | 8 | Work was started on the adding contributions workflow. The existing workflow involves 9 | manual addition of new libraries to the data files of the repository 10 | `processing-contributions`. The new work refactors this repository. There are two main 11 | aspects to this work - converting the data store from the current configuration files 12 | to a consolidated yaml file, and setting up an automated workflow that creates the pull 13 | request with the new contribution added to the database file. 14 | 15 | The work accomplished this month on this workflow was 16 | 17 | 1. the creation of an initial version of the database yaml file 18 | 2. scripts that convert the data in the yaml file into the file formats needed for the 19 | contribution manager, and the website 20 | 3. an issue form, where new contributors input the url to their released properties file 21 | 4. a workflow triggered by the issue form, that retrieves the properties, edits the 22 | database file, and create a pull request. 23 | 24 | ### Challenges Encountered 25 | 26 | It was an interesting challenge, to submit a graphic to go along with the 27 | newsletter announcement. My initial attempts at manual graphic design were rather 28 | terrible. I'd like to think that the graphic I eventually submitted, coded in Processing, 29 | was a considerable improvement. 30 | 31 | In hindsight, I'd say a previous challenge was balancing work with documentation. 32 | This month, I allowed myself the space to write and sort out the brain, but I just 33 | wanted to highlight that it was a challenge previously. 34 | 35 | 36 | ### October Plans 37 | 38 | The plans in October are: 39 | 40 | 1. Finish work on the add contributions workflow. 41 | 2. Advertise the new library template more broadly, for instance in the discourse, 42 | discord for Processing, and writing an article. 43 | 3. Document the add contributions workflow 44 | 4. To refactor the Processing library wiki pages - these could be wiki pages for 45 | the template, for the processing repo, or on the website -------------------------------------------------------------------------------- /2024_NewBeginnings/monthly-reports/pr05_2024_July_report_Dora_Do.md: -------------------------------------------------------------------------------- 1 | # July monthly report: Prototyping a collaborative code editor for Processing 2 | By Dora Do 3 | 4 | ## Completed Tasks 5 | 6 | The focus of the first month of this project was about research, technical feasibility, and project scoping. 7 | 8 | - Created a [Github repo](https://github.com/doradocodes/processing-collab-editor) for this project and [Github Project](https://github.com/users/doradocodes/projects/1) to track tasks and progress 9 | - Created a prototype of an Electron app that can run Java processes and display logs from the process on the app 10 | - Create a [MVP](https://docs.google.com/document/d/1rbMAvGepee8lPmaNjLItSL4Ah5MW0jLUfjhWWO8HgyQ/edit?usp=sharing) 11 | - Researched different frameworks (ie. Theia, CodeMirror, JupyterLab, VSCodium) and tools (ie. yjs, CodeMirror collab editing) 12 | - Created a test project of Theia and experimented with customization 13 | - Created a test project of an Electron app with CodeMirror 14 | - Create a [Figma file](https://www.figma.com/design/SxmAa94lAP37OLPpY1hgVf/Processing-Collaborative-Editor-(PCE)?node-id=5-21&t=4mR3IQXoh30vJv6m-1) with initial wireframes and inspiration gathering 15 | 16 | ## Challenges 17 | 18 | - Testing some of the frameworks to see if they were customizable to our needs took a while. I ran into many errors trying to run the projects 19 | - Through researching the different options, we are still having a hard time choosing which direction to go since each have their own pros/cons 20 | - There are a lot of ideas about reimagining editor, versus matching the functionality of the current Processing IDE. There are many different directions for how this IDE could be used, and determining which audience or type of user that we want to target 21 | 22 | ## Upcoming plans 23 | 24 | - By the end of the first week of August, we should choose a direction for implementation, whether it be using an existing framework like Theia, or doing a custom build. During our next cohort meeting, I will be sharing out the findings from my research and to get the opinions of the cohort members, especially since some have experience with libraries we are thinking of using, like CodeMirror 25 | - While we decide on the direction of how this prototype should be built, I will also start creating wireframe sketches of how the editor would look, and start getting feedback on those designs 26 | - Once the designs are down, I will start breaking down MVP into tasks, and tracking them on Github Projects 27 | -------------------------------------------------------------------------------- /2024_NewBeginnings/monthly-reports/pr05_2024_August_report_Dora_Do.md: -------------------------------------------------------------------------------- 1 | # August monthly report: Prototyping a collaborative code editor for Processing 2 | By Dora Do 3 | 4 | ## Project Overview 5 | 6 | This month, we made significant progress on the Processing Collaborative Editor (PCE) project. Our focus was on evaluating technology options, designing the user interface, and beginning the implementation of core features. 7 | 8 | ## Key Decisions and Progress 9 | 10 | - Technology Stack: 11 | - After careful consideration and discussions with mentors, we decided to proceed with a custom build using CodeMirror rather than Theia. 12 | - This decision was based on CodeMirror's ease of implementation, existing collaborative features, and faster development potential. 13 | - User Interface Design: 14 | - Created wireframes and a Figma prototype for the PCE UI. 15 | - The design features a two-column layout with collapsible sections for improved usability. 16 | - Development Progress: 17 | - Successfully integrated CodeMirror editor with Electron. 18 | - Implemented basic functionality to run Processing sketches from within the application. 19 | - Began integration of CodeMirror's collaborative features. 20 | 21 | ## Challenges and Solutions 22 | 23 | We encountered some technical challenges, particularly with running the `processing-java` process when using Vite. This was resolved by adjusting our build process to use Vite only for the React app portion. 24 | 25 | ## Collaborative Features 26 | 27 | We're exploring two main approaches for implementing real-time collaboration: 28 | 29 | 1. Using a locally hosted intermediary server with web sockets 30 | 2. Using peer-to-peer connections with WebRTC 31 | 32 | Each approach has its pros and cons, and we're currently evaluating which will be the best fit for our project goals. 33 | 34 | ## Next Steps 35 | 36 | - Complete the integration of CodeMirror's collaborative features 37 | - Refine the UI design and implement remaining layout components 38 | - Decide on and implement the collaborative connection approach 39 | - Begin work on integrating Language Server Protocol (LSP) for enhanced coding features 40 | 41 | ## Conclusion 42 | 43 | August has been a productive month for the PCE project. We've made critical decisions on our technology stack, advanced our UI design, and begun implementing core features. As we move into September, our focus will be on refining the collaborative aspects of the editor and continuing to improve the overall user experience. 44 | -------------------------------------------------------------------------------- /2024_NewBeginnings/monthly-reports/pr05_2024_July_report_Nahee_Kim.md: -------------------------------------------------------------------------------- 1 | # July Progress Report 2 | 3 | ## Project Title: Enhancing the p5.js Editor with CodeMirror 6 and React Functional Components 4 | 5 | ### Project Description 6 | Upgrade the CodeMirror component in the p5.js Editor from Version 5 to Version 6, implementing enhanced features for accessibility, mobile support, internationalization, and other editor improvements. This will also require refactoring related components from class-based to functional components using React. 7 | 8 | ### Execution Plan 9 | 1. **CodeMirror 5 to 6 Transition** 10 | - Identify and import necessary sub-packages under the @codemirror scope. 11 | - Refactor existing methods into transactions and dispatch them collectively. 12 | 2. **React Component Refactor** 13 | - Convert class-based React components to functional components. 14 | - Use tools like ChatGPT for initial translations, build and run the code to identify necessary corrections. 15 | 3. **TypeScript Conversion** 16 | - Convert JavaScript-based codebase to TypeScript for files that require immediate changes. 17 | - Understand variable and function definitions through TypeScript to improve development efficiency. 18 | 19 | ### Timeline 20 | **Month 1: Planning and Initial Setup** 21 | * **Week 1-2:** 22 | - Review the existing p5.js codebase. 23 | - Set up development environment. 24 | - Identify key components to be refactored and upgraded. 25 | - Start setting epics and adding tickets on Zenhub board 26 | * **Week 3-4:** 27 | - Begin the conversion of simple class components to functional components. 28 | - Start importing necessary sub-packages for CodeMirror 6. 29 | 30 | ## Progress for July 31 | 32 | ### Completed Tasks 33 | - Synced the personal forked repo with the most updated p5.js editor repo. 34 | - Met with the mentor Connie on July 5th, 2024. 35 | - Introduced ourselves to each other. 36 | - Set up the bi-weekly meeting schedule (Monday 8 PM EST). 37 | - Discussed scope, timeline, and conventions (synchronize prettier style with the original p5 editor branch). 38 | - Created a project plan document. 39 | 40 | ### Challenges Encountered 41 | - Initial synchronization of the forked repository took longer than expected due to merge conflicts and dependency updates. 42 | 43 | ### Plans for August 44 | - **Week 1-2:** 45 | - Continue conversion of simple class components to functional components. 46 | - Begin importing necessary sub-packages for CodeMirror 6. 47 | - Set up initial CodeMirror transactions. 48 | - **Week 3-4:** 49 | - Test and debug the refactored components. 50 | - Start the integration of CodeMirror 6 with the p5.js Editor. 51 | 52 | ### Note 53 | - I couldn't make significant progress in the last week of July due to personal reasons. -------------------------------------------------------------------------------- /2024_NewBeginnings/monthly-reports/pr05_2024_August_report_Nahee_Kim.md: -------------------------------------------------------------------------------- 1 | # Monthly Report - August 2024 2 | 3 | **Overview:** This month, I focused on resolving issues related to the `useEffect` dependency array and researched ways to improve the integration of `@uiw/react-codemirror` with React lifecycle events, particularly in the context of CodeMirror 6. Here’s a detailed breakdown of progress: 4 | 5 | ## 1. Resolving useEffect Dependency Array Issues: 6 | 7 | **Identifying the Issue:** 8 | I identified that most of the issues were stemming from the way the dependency array in `useEffect` was structured. This required further investigation to determine which variables could safely be removed from the array without affecting functionality. 9 | 10 | **Proposed Solutions:** 11 | 12 | Connie’s Idea: Connie suggested separating `useEffect` hooks as much as possible, so each one has its own relevant dependency array. This would minimize unnecessary re-renders and make the code cleaner. 13 | Additionally, Connie proposed that leaving the dependency array empty in cases where the `useEffect` acts like `componentDidMount` seemed to alleviate some issues, particularly around `tidyCode`. This needs further testing to confirm stability. 14 | 15 | ## 2. Researching and Testing @uiw/react-codemirror: 16 | 17 | **Goal:** 18 | My goal was to explore how to better incorporate React’s lifecycle with CodeMirror 6, improving overall integration and reducing potential issues. 19 | 20 | **Current Progress:** 21 | 22 | I worked on converting the `Editor/index.jsx` file using `@uiw/react-codemirror` and tested this integration extensively. A branch/PR has been created for tracking the changes: 23 | PR link 24 | 25 | **Key Changes Implemented:** 26 | 27 | Switched to using `EditorView` along with a editorView state in React instead of relying on `cmInstance.current`. 28 | Adopted `view.dispatch` for applying changes to the editor view, replacing the previous method of calling functions directly from the instance. 29 | Modified the usage of linter and Emmet to align with CodeMirror 6 updates. 30 | 31 | **Next Steps:** 32 | I plan to create a custom hook (`useCodemirror`) to streamline the setup process for the CodeMirror view, which will make the code more maintainable and reusable. 33 | 34 | ## 3. React Functional Component Conversion: 35 | 36 | **Changes Applied:** 37 | I completed the React functional component conversion for the following components: `CollectionList`, `SketchList`, `FileNode`, `ConsoleInput`, and `AssetList`. This was the last conversion ticket in the project, marking the completion of this task. 38 | 39 | **Looking Forward:** In the coming month, I will continue refining the use of the `@uiw/react-codemirror` library and conduct additional testing to ensure smooth integration. I will also explore further improvements to the `useEffect` implementation to optimize performance and stability in the application. 40 | -------------------------------------------------------------------------------- /2024_NewBeginnings/monthly-reports/pr05_2024_August_report_Claudine_Chen.md: -------------------------------------------------------------------------------- 1 | # August Monthly Report - for Simplifying the Workflow for Libraries, Tools, and Modes 2 | ## Claudine Chen 3 | 4 | ### Work Accomplished 5 | 6 | Work continued to finalize the library template. 7 | 1. This included debugging Gradle tasks, and refining flows and comments in 8 | the `build.gradle.kts` for usability and comprehension. 9 | 2. Also, the example library was extended to include an external dependency, 10 | to ensure that the build task worked. 11 | 3. MkDocs support was added. There is a starter website, and a Github workflow 12 | that deploys the website as a Github Pages project website. This starter website 13 | also documents the library template. 14 | 15 | We also started planning and designing the adding contribution workflow. 16 | In a new repo, the existing contributions source files will be processed into 17 | a new database file, in yaml format. This data will then be processed into 18 | the artifacts needed by the website and contributions manager. 19 | New contributions will come in as an issue via form, which will configure and 20 | create a new branch and pull request, with the contribution added to the 21 | yaml database file. 22 | This work will begin in September. 23 | 24 | 25 | ### Challenges Encountered 26 | One goal that in the end was cut from the template, was returning the version of the library as a library method, 27 | and synchronizing that version with gradle. To do this, either the user must maintain the version manually, or it 28 | could be synchronized via code. This was synchronized previously by inputting this value into the `library.properties` file, 29 | and propagating that value into the code with a search and replace script. Another solution could be to read in 30 | a properties file and return the value; this would require the properties file to be part of the library. Other 31 | ways were explored, to see if the Gradle file could be made aware of the Java method. In the end, it was decided 32 | not to pursue this further, since the need for the version to be reported by the library would only be in cases of 33 | debugging an error, so hopefully very rarely. We are also aware that any file or structure included in the template 34 | might be regarded as required, so we only want to offer what is required in the template. 35 | 36 | 37 | ### September Plans 38 | 39 | The plans in September are 40 | 1. to release the library template, on Discourse, and to invite feedback. 41 | 2. continue work on the add contributions flow. 42 | 1. create a new repo, and process the source data of processing-contributions into a new yaml database file. 43 | 2. create scripts that export the database into artifacts for the website 44 | 3. create scripts that export the database into artifacts for the contribution manager 45 | 4. create the workflow, such as the issue form template, and the automatic creation of the pull request 46 | -------------------------------------------------------------------------------- /2024_NewBeginnings/monthly-reports/pr05_2024_July_report_Claudine_Chen.md: -------------------------------------------------------------------------------- 1 | # July Monthly Report - for Simplifying the Workflow for Libraries, Tools, and Modes 2 | ## Claudine Chen 3 | 4 | ### Work Accomplished 5 | The main goal this month was to make initial strides in developing a library template using Gradle. Initial work was to explore what work had already been done by the community, and to understand what each repo did, and how they did it. From this understanding, a template that uses the best-of ideas is being assembled at https://github.com/mingness/processing-library-template. 6 | This template has been the collaborative work of myself and Katsuya Endoh, with helpful guidance from Stef Tervelde and Raphaël de Courville. 7 | 8 | Once work on the template is completed at this stage, we will have completed the following expected outcomes: 9 | - Repository templates for libraries using the new Gradle templates. 10 | - Automation for building and exporting artifacts, including PDEX/PDEZ bundles. 11 | - CI/CD deployment of the reference to GitHub pages including install links using the new pdez:// and pdex:// protocols. 12 | 13 | The main challenge met in this work was working with Gradle. The documentation has not been clarifying with respect to what plugins specifically do, and I would have to do extra searches to find out how to implement something using Kotlin, the new default and our chosen language, instead of Groovy, the previous long-time standard. Also, I had unexpected difficulties building from the commandline - the commandline build would break for situations that would build successfully from the Gradle menu in Intellij. 14 | 15 | Also, in an effort to create a process and resources that are intuitive and address the communities needs, a handful of people have been interviewed thus far about their own experiences developing libraries, which suggests some takeaways: 16 | - Most people expected documentation for building libraries to be on the website. This is a good idea, because not everyone knows to explore Github for information. 17 | - People eventually found resources using internet search. The information found and path followed varied by individual. This suggests existing information and resources are fractured and don't lead to predictable outcomes. 18 | - Needing to go to Processing to test the library grew tiresome after a while. Some would like a way to test library installation within intellij 19 | - The processing hosted gradle template is designed for the developer to interact with a build.properties file. While users found the template worked and was well documented, some didn't realize that dependencies were also to be listed in the build.properties file, and they interacted with the gradle file to add dependencies. This reminds us that users will not always interact with a design as expected. 20 | - Some people struggled with javadocs, and instead opted for mkdocs, which creates pages using markdown. 21 | - Intellij is a featureful IDE that is good for beginners. 22 | 23 | ### August Plans 24 | The plans for next month are twofold: 25 | 1. I plan to finish the initial draft of the library template, and then release it for testing. I would like to invite developers to test the new template at https://github.com/mingness/processing-library-template, but also the previous template at https://github.com/processing/processing-library-template-gradle. I will solicit feedback on their experiences. 26 | 2. Design and start the implementation of a more automated, simplified workflow for adding contributions to the processing website and contributions manager. 27 | 28 | 29 | -------------------------------------------------------------------------------- /2024_NewBeginnings/monthly-reports/pr05_2024_July_report_Diya_Solanki.md: -------------------------------------------------------------------------------- 1 | # July monthly report: Develop a VSCode LSP Server Extension for Processing 2 | By Diya Solanki 3 | 4 | - [Project repository](https://github.com/diyaayay/processing-language-server-extension/) 5 | - [Working branch](https://github.com/diyaayay/processing-language-server-extension/blob/main) 6 | 7 | 8 | ## Completed Tasks (what's done so far) 9 | 10 | - [x] Reviewed [Efratror's prior work](https://github.com/efratror/LS4P) on the LSP for Processing and explored the possibility of updating to Processing 4.1+ and 4.3 (that one works on `Processing 4.0.8b`) 11 | - [x] Built Efratror's LSP for `Processing 4.0.8b` using `Java 8` and the deprecated VSCode language server extension protocol. 12 | - [x] Set up a roadmap to track tasks and decide on the approach for the project. 13 | - [x] Successfully transitioned to Processing 4.3 by generating a release zip file containing `core.jar` and setting up the JRE zip files for running Processing sketches locally. 14 | - [x] Established that the LSP is running locally for all Processing sketches and began discussions to find the latest versions of classes required for `custom.jar`. 15 | - [x] Updated all NPM dependencies to use the latest packages. 16 | - [x] Using `JDK 17.0.8` and `node v20.15.1` (the current LTS) 17 | - [x] Managed to make the working version of the new transition! (but I still need to figure out the most concrete way of integrating the latest release and that is mainly for the ending part of the program so that is being researched simultaneously!) 18 | 19 | 20 | ## Challenges Encountered 21 | 22 | - Transitioning from `Processing 4.0.8b` to `Processing 4.3` was more complex than anticipated, requiring adjustments in dependencies and approach. 23 | - The `custom.jar` file in the old project [repo](https://github.com/efratror/LS4P) configuration involves significant changes from earlier versions, and specific classes need to be handpicked and updated. 24 | - In the latest version of processing, there are significant changes, e.g., `javafx` is not shipped with the core library now (it used to, earlier) so I compiled the new `javafx` (built it with `ant`, [reference repo here](https://github.com/processing/processing4-javafx/releases)) and integrated it with the latest `core.jar`. 25 | - I was facing some path issues also (probably due to new JRE) that I resolved. 26 | - I needed new `JRE` to make it compatible with the latest processing so I compiled it with the latest `core.jar` using [this repository](https://github.com/efratror/LS4P). 27 | - The NPM Java package currently used is optimized for OpenJDK 7, whereas `Processing 4.3` works best with `JDK 17`, causing compatibility issues. 28 | 29 | 30 | 31 | ## Plans for August 32 | 33 | - [ ] Now that `Processing 4.3` is integrated (as in running all kinds of sketches), the focus for August would be to figure out the most concrete way of integrating the latest release to be built on any system and build LSP features on the processing sketch document using the latest VSCode LSP extension API like code completion, hover, go-to definition, diagnostics, validate sketches, etc. 34 | - [x] The priority of choosing the features would be to start from the most essential features and release a version for testing. (Update 02/Sep/2024: I now have a [roadmap](https://github.com/diyaayay/processing-language-server-extension/blob/main/ROADMAP.md) for the project that has a tentative list of tasks!) 35 | - [x] August would also be focussed on integrating the `Processing 4.3` specific keywords in the AST which is the key to developing the the features on the text Document. 36 | - [x] Other features such as `Run and Stop` buttons in the extension host for ease of use. -------------------------------------------------------------------------------- /2024_NewBeginnings/monthly-reports/pr05_2024_September_report_Miaoye_Que.md: -------------------------------------------------------------------------------- 1 | # September Monthly Report: Revamp the Friendly Error System (FES) in p5.js 2 | 3 | By Miaoye Que, mentored by Kenneth Lim and Dave Pagurek with support from Qianqian Ye. 4 | 5 | ## Work Completed 6 | 7 | ### Accidentally Overridden Constants and Functions 8 | 9 | To see all PRs and real-time progress for accidentally overriden constants and functions, check out the [master issue #7269](https://github.com/processing/p5.js/issues/7269). 10 | 11 | - [x] Started a new `sketch_verifier` file that will detect accidentally overriden constants and functions. 12 | - [x] Script can now retrieve user's code as a string and with the help of `Espree` (parser), identify user-defined functions and constants. 13 | 14 | ### Parameter Validation 15 | 16 | To see all PRs and real-time progress for parameter validation, check out the [master issue #7178](https://github.com/processing/p5.js/issues/7178). 17 | 18 | - [x] Parameter validation now validates against real p5 (i.e. `p5.Color`) and web API objects (i.e. `KeyboardEvent`). 19 | - [x] Updated `convert.js`, script that generates `parameterData.json` so that 1) we ignore false definitions and 2) deduplicate overloads with the same order and types. 20 | - [x] Implemented the new modular syntax and dependency injection pattern for parameter validation. 21 | - [x] Parameter validation now handles array-type arguments dynamically, i.e. identifies that an argument's type ends with `[]` and applies `z.array()` to object type. 22 | - [x] Instead of using `zod-validation-error` library, now prints error messages with custom logic catered towards beginners. 23 | - [x] Updated test file to cover all test cases from the old test file and then some. 24 | - [x] Added parameter validation support for paletteLerp, see [#6960](https://github.com/processing/p5.js/issues/6960). 25 | - [x] Some code cleanups now that the parameter validation is almost at completion. 26 | 27 | ## Challenges Encountered 28 | 29 | ### General 30 | 31 | - Again, I struggled with time management because I started a full-time job early September. I was working at a pretty erratic cadence. Fortunately, I'm only a little behind schedule so this was not a major concern. 32 | 33 | ### Accidentally Overridden Constants and Functions 34 | 35 | - Had trouble getting the user's code as a string and needed help from Ken and Dave. 36 | 37 | ### Parameter Validation 38 | 39 | - I don't think I faced challenges for parameter validation per se, but what I thought would be done swiftly turned out to require more of my care and attention, and follow-up issues kept emerging. For example, after I finished the initial implementation, we realized that the messages printed were not beginner-friendly because of how technical they were. So the approach I thought would save me time (leveraging on an existing library `zod-validation-error`) turned out to be inappropriate for the task, and custom logic was required after all. 40 | 41 | ## Remaining Tasks 42 | 43 | ### Accidentally Overridden Constants and Functions 44 | 45 | - [ ] Check if user-defined functions and variables have already been defined in `p5` and shouldn't be re-defined. 46 | - [ ] Generate friendly messages for re-definitions that shouldn't take place. 47 | - [ ] Write comprehensive testing for the feature. 48 | - [ ] Integrate it into the `dev-2.0` branch. 49 | 50 | ### Parameter Validation 51 | 52 | - [ ] Update friendly messages again if necessary. 53 | - [ ] Integrate it into the `dev-2.0` branch. 54 | 55 | ## October Plans 56 | 57 | - Wrap up the implementation, testing, and integration for both priorities. 58 | - Update documentation, wherever applicable. 59 | - Prepare for project wrap-up and presentation, i.e. interview with Yoon, the final group presentation. 60 | -------------------------------------------------------------------------------- /2024_NewBeginnings/monthly-reports/pr05_2024_September_report_Dora_Do.md: -------------------------------------------------------------------------------- 1 | # September monthly report: Prototyping a collaborative code editor for Processing 2 | By Dora Do 3 | 4 | ## Project Overview 5 | 6 | This month focused on refining the collaborative editing functionality of our Electron-based code editor, designed for Processing sketches. We explored different real-time collaboration frameworks, worked on packaging the app for distribution, and continued improving the user interface. Notable progress was made in integrating YJS with CodeMirror for real-time collaboration and addressing the challenges of running processing-java within the app across different operating systems. 7 | 8 | ## Key Decisions and Progress 9 | 10 | We decided to use WebRTC for small groups (1-3 users) but identified its limitations in larger groups due to performance issues. Therefore, we are leaning towards WebSockets for text-based messaging due to their reliability and better support for scaling. We successfully integrated the YJS framework with the CodeMirror editor and tested it with WebSocket providers. The Radix UI component library was also integrated into the app to enhance the overall user interface and provide a consistent design system for future contributors. 11 | 12 | On the packaging front, we used `electron-builder` to package the app and have it running from the Applications folder. Some issues with paths required adjustments to the `package.json` and asset paths, but this was resolved, allowing for successful packaging. 13 | 14 | ## Challenges and Solutions 15 | 16 | - Scaling WebRTC: 17 | - We realized WebRTC could not handle larger groups effectively due to performance issues. We are now focused on WebSockets, which are more stable for our use case. 18 | - Handling initial state sync in collaborative sessions: 19 | - We faced a problem where new clients joining a collaborative sketch were not receiving the initial document state. The solution involved setting the entire document upon room creation, allowing new clients to receive the correct state. 20 | - Running Processing sketches in the packaged app: 21 | - Running the `processing-java` executable in the packaged app posed issues due to hardcoded paths for the Java library and JDK. The solution we implemented was a Settings popup where users can manually select the path to their Processing installation. This is a temporary workaround until more automated solutions, such as Stef's headless version of Processing, are available 22 | - Error handling when clients join non-existent sketches: 23 | - There was an issue with properly handling errors when clients try to join a sketch that doesn't exist. We're currently improving error handling to detect and notify users when they attempt to join an invalid sketch. 24 | 25 | ## Next Steps 26 | 27 | 1. **Further Testing**: We need to conduct more extensive tests on both macOS and Windows, especially ensuring that the `processing-java` executable works across platforms. 28 | 2. **Improve User Onboarding**: I plan to implement an onboarding flow that will guide users through setting up their Processing environment. This will include selecting their Processing installation path and verifying that their system is ready to run sketches. 29 | 3. **Enhance UI and UX**: Based on Ted’s feedback, we redesigned the side navigation and improved the sketch creation workflow. Further refinement is needed for a more intuitive user experience. 30 | 4. **Fixing Known Bugs**: Continue to refine error handling for clients joining invalid rooms and improve document state synchronization in collaborative sessions. 31 | 32 | ## Conclusion 33 | 34 | This month saw significant progress in real-time collaboration, UI refinement, and resolving packaging issues for app distribution. While there are still challenges, especially around running Processing sketches and scaling WebRTC, we've implemented viable solutions for the next phase of testing. With more rigorous testing planned, we're on track to deliver a stable prototype for broader feedback and iteration. 35 | -------------------------------------------------------------------------------- /2025_BuildingBridges/monthly-reports/pr05_2025_July_report_Stephan_Max.md: -------------------------------------------------------------------------------- 1 | # Monthly Report – July: Desktop Support for p5.js via a New Processing Mode 2 | 3 | ## Overview 4 | 5 | This month was all about learning the necessary skills for this project: brushing up on Java; learning Kotlin with a focus on its differences compared to Java, Java interoperability, and object-oriented programming; and growing familiar with Gradle. There is still a lot to learn, but I have been feeling confident enough to make my dent in the Processing code base. For that we decided to integrate [Stef’s proof-of-concept p5.js mode](https://github.com/Stefterv/processing-p5js-mode) into Processing as a second core mode next to the default Java one. Apart from this migration, milestone 1 also included communication between a running sketch inside Electron and the Processing Develoment Environment (PDE). I achieved this one-way from Electron to the PDE. p5.js sketches can now be run out of the PDE and Electron’s console output is mirrored to the PDE console. 6 | 7 | ![p5.js sketch running inside PDE, with debug output on the console.](./pr05_2025_July_report_Stephan_Max.png) 8 | 9 | ## Key Decisions and Progress 10 | 11 | We decided to stick with the technology—Electron as the internal Browser engine to render p5.js sketches—established by Stef’s PoC. No other decisions have been necessary so far. I am using [Electron’s Inter-Process Communication](https://www.electronjs.org/docs/latest/tutorial/ipc) capabilities to pipe console output of the sketch rendering process back into the PDE console. 12 | 13 | ## Challenges and Solutions 14 | 15 | There were no “hard” challenges regarding used technology, build setup, or code base literacy. My challenges and learnings this month rather lean towards the personal side and character traits. I know that I am prone to going depth-first, rather than breadth-first, which means I tend to get lost in details of very specific areas of the code instead of making sure that a functional vertical slice of the project is present and moved forward. Even though I tried my best to have a demonstrable version of the p5.js mode ready, I became easy prey to the inner workings of Gradle build scripts and core application classes like `Base`, `Editor`, and `Mode`. 16 | 17 | The solution to this is my attentive mentor, Stef, who helped me steer back on track. I also decided to maintain a [project wiki](https://github.com/stephanmax/processing4/wiki) where I can get my learnings out on paper and remove the occupying attention residue. 18 | 19 | Due to a sick leave of one week, I am a bit behind schedule. However, I invested some time into rearranging competing workload and am confident to make up for the lost time in the first half of August. 20 | 21 | ## Next Steps 22 | 23 | I plan to improve the user experience of the p5.js mode and pave the way for npm package handling. This includes two-way communication between PDE and sketch (react to canvas size changes, hot reloading, debugger), integration of the Node runtime, move HTML and JavaScript resources needed by Electron into a dedicated space to allow the robust (re-)execution of multiple sketches. 24 | 25 | ## Relevant Resources 26 | 27 | - [Project repository](https://github.com/stephanmax/processing4/tree/main) (Processing 4 fork) 28 | - [Working branch](https://github.com/stephanmax/processing4/tree/pr05-poc) 29 | - [Project wiki](https://github.com/stephanmax/processing4/wiki) (everything I deem worthy dumped on the homepage for now to avoid premature content organization; will improve structure as I go) 30 | 31 | ### Recommended Learning 32 | 33 | - [Programming Kotlin](https://pragprog.com/titles/vskotlin/programming-kotlin/) by Venkat Subramaniam 34 | - [Build Java Software With Gradle Like a Pro (Full Gradle Course)](https://www.youtube.com/watch?v=SJKXJHKTRX8) by Tom Gregory (build script DSL of choice if Groovy, not Kotlin; some things are out of data; still: core content is valid and helps building lasting mental models) 35 | - [Stef’s Gradle Runner PR for a new internal build system](https://github.com/Stefterv/processing4/pull/7) 36 | -------------------------------------------------------------------------------- /2024_NewBeginnings/monthly-reports/pr05_2024_August_report_Miaoye_Que.md: -------------------------------------------------------------------------------- 1 | # August Monthly Report: Revamp the Friendly Error System (FES) in p5.js 2 | 3 | By Miaoye Que, mentored by Kenneth Lim and Dave Pagurek with support from Qianqian Ye. 4 | 5 | ## Work Completed 6 | 7 | ### Accidentally Overridden Constants and Functions 8 | 9 | - [x] Picked out a parser for detecting overridden constants and functions (`Espree`). 10 | 11 | ### Parameter Validation 12 | 13 | To see all PRs and real-time progress for parameter validation, check out the [master issue #7178](https://github.com/processing/p5.js/issues/7178). 14 | 15 | - [x] Start a new parameter validation file using Zod for p5.js. 16 | - [x] Update script that generates `parameterData.json` to further reduce the size of the file. 17 | - [x] Validate against real p5 constants. 18 | - [x] Handle schema generation for optional parameters. 19 | - [x] Implement a distance calculation mechanism similar to [`scoreOverload`](https://github.com/processing/p5.js/blob/ada3c05da35bb8daf3b9d0ca1dc5713bff0b95c1/src/core/friendly_errors/validate_params.js#L425) that gives users the proper error message when parameter validation fails. 20 | - [x] Start a test file for new parameter validator. 21 | - [x] Fix the case where a parameter can be a mix of constants and primitive types. 22 | 23 | ## Challenges Encountered 24 | 25 | ### General 26 | 27 | - At the end of July, I planned to set August as my implementation-heavy month, with the aim of getting both of the priorities for the FES project implemented or almost implemented. However, I ended up spending most of my time getting TypeScript exporting to work. Later, I realized that some parts of my plan, although conceptually ideal, might be more complicated and uncertain than I expected when it comes to implementation. To this end, I'm letting myself adjust my timeline a bit, to be more realistic with the amount of work that needs to be done. 28 | - More generally, I initially divided my grant period into clearly-defined, task-oriented milestones (i.e. research first, then implementation, ending with testing and integration). In reality, the process of coding and implementation often reveals oversight and issues that weren't apparent during the research phase. For future projects (both mine and other Processing Foundation associates'), it's better to envision grant periods in terms of _iterations_, instead of _phases_. 29 | 30 | ### Accidentally Overridden Constants and Functions 31 | 32 | - I spent minimal time on this task in July, so there's nothing much to report here. 33 | 34 | ### Parameter Validation 35 | 36 | - Approach: leverage on `zod`, a TypeScript-first schema validation library with static type inference. 37 | - Planned workflow: generate `.d.ts` file -> use the generated file as input in `ts-to-zod`, generate `zod` schema -> use the generated schema for parameter validation. 38 | - `.d.ts` file generated is currently too big and has many duplicated elements. 39 | - Methods that are chainable and return the p5 instance inline an interface for the whole p5 module as its return type. 40 | - Unsure if we can get clean TypeScript types; this might be a dead end that we simply haven't figured out yet, and it'd require a lot of work to have a clear answer. 41 | - New workflow: based on the JSON file that contains all the p5 methods and their accepted parameters, write a custom parser that generates `zod` schema for validation. 42 | - Writing a custom parser means that we need to explicitly define each parameter type and constant, which turned out to be rather complicated and requires a lot of attention to detail. 43 | - For object (both p5 objects such as `p5.Camera` and Web API objects such as `Audionode`) validation to work, I need to bring up an actual browser environment, which is more complicated than just runnign `node file.js` for testing. 44 | 45 | ## September Plans 46 | 47 | - Wrap up the implementation, testing, and integration for parameter validation. 48 | - Start working on the next priority: report accidentally overridden constants and functions; at least get to a 75% done state. 49 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # `pr05` Code of Conduct 2 | 3 | ## Who We Are 4 | 5 | The Processing Foundation is a nonprofit whose mission is to promote 6 | software literacy in the arts and visual literacy in the field of technology. 7 | 8 | Our efforts also aim to support people of all backgrounds and actively address 9 | systemic biases in art and tech spaces, in particular, those related to gender, 10 | race, class, and accessibility. 11 | 12 | This means that we: 13 | 14 | * Welcome contributors at any stage of their programming journey, from newcomers to professionals. 15 | * Do not assume knowledge or imply there are things that somebody should know. 16 | * Understand that people are the experts of their own experiences, and bring a variety of valid perspectives to the development of open source projects. 17 | * Know that contribution is not just limited to code, and can also include working on documentation, filing issues and bug reports, and other important forms of input. 18 | * Work to offer help and guidance when we are able to do so. 19 | 20 | ## Our Standards 21 | 22 | All grantees, mentors, advisors, and people involved in the `pr05` program agree to abide by the following code of conduct. 23 | 24 | ### Respect other people 25 | 26 | Treat other community members with respect and kindness. Examples of behavior that contributes to a positive environment for our community include: 27 | 28 | * Demonstrating empathy toward other people, and assuming best intentions; 29 | * Being mindful in how we approach and interact with others; 30 | * Being respectful of differing opinions, viewpoints, and experiences; 31 | * Listening in times of conflict, and communicating as best we can while acknowledging each other's feelings; 32 | * Giving and gracefully accepting constructive feedback; 33 | * Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience; 34 | * Making space for everyone to be heard. 35 | 36 | ### Be mindful of how you engage with others 37 | 38 | The following behavior will not be tolerated: 39 | 40 | * The use of sexualized or violent language or imagery; 41 | * Unwelcome sexual attention or advances of any kind; 42 | * Offensive comments related to race, class, gender identity and expression, 43 | neuro-type, ability, religion and culture, age, skill level, occupation, background, or political opinion; 44 | * Public or private harassment; 45 | * Publishing others' private information, such as a physical or email address, without their explicit permission; 46 | * Any other conduct which could reasonably be considered inappropriate in a professional setting. 47 | 48 | When in doubt, if it could make people uncomfortable, don't do it. 49 | 50 | ### Be open-minded 51 | 52 | We understand that technology changes all the time, and that contributors may bring new or different views based on their own experiences. Be open-minded when engaging with other people and their ideas, and embrace the new or unfamiliar as opportunities for positive change. 53 | 54 | ### Be considerate 55 | 56 | We focus on what is best not just for us as individuals, but for the health of the overall community. Be considerate of other people, and think about how your contributions will affect the community as a whole. 57 | 58 | 59 | ## Enforcement 60 | 61 | If you believe someone is violating the Code of Conduct, we ask that you report it by emailing safety@processing.org. Please include your name and a description of the incident, and we will get back to you as soon as possible. 62 | 63 | Sometimes, participants violating the Code of Conduct are unaware that their behavior is harmful, and an open conversation clears things up to move forward. However, if a participant continues with the behavior, the Processing Foundation team may take any action they deem appropriate, up to and including expulsion from all community spaces and identification of the participant as a harasser to other members or the general public. 64 | 65 | ## Attribution 66 | 67 | This Code of Conduct is adapted from the [Contributor Covenant version 2.1](https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1]); the [p5.js Code of Conduct](https://github.com/processing/p5.js/blob/main/CODE_OF_CONDUCT.md); and the [ml5.js Code of Conduct](https://github.com/ml5js/Code-of-Conduct). 68 | -------------------------------------------------------------------------------- /2025_BuildingBridges/monthly-reports/pr05_2025_August_report_Stephan_Max.md: -------------------------------------------------------------------------------- 1 | # Monthly Report – August: Desktop Support for p5.js via a New Processing Mode 2 | 3 | ## Overview 4 | 5 | The goal of this month was to turn the proof of concept into a proper mode with the capability to install arbitrary `npm` packages. This included putting all Electron scaffolding—HTML and JavaScript code—as files into a sub-folder of the project rather than hard-coding it in Kotlin. Gradle allowed me to copy them over into a resources folder of Processing and make them accessible during runtime. I also pieced together a simple user interface to search [npm’s registry](https://registry.npmjs.org) for packages and install them at the touch of a button. This allows p5.js sketch authors to leverage not only Node’s [standard library](https://nodejs.org/docs/latest/api/) but also the vast landscape of third-party Node packages. Saving each frame of an animated sketch anywhere in the file system via Node’s `fs` and `path` modules hopefully demonstrates the potential of this project. 6 | 7 | ![p5.js sketch running in PDE, each frame saved as png screenshot in user’s home directory thanks to Node’s filesystem library.](./pr05_2025_August_report_Stephan_Max.png) 8 | 9 | ## Key Decisions and Progress 10 | 11 | We do not want to lock sketch authors in the Processing Development Environment, but allow them to work on their p5.js code in whatever environment they feel comfortable in. Hence we made it a requirement that every p5.js sketch created in Processing is just a collection of JavaScript and HTML files that can be built (`npm install`) and run (`npx electron .`) with common JavaScript tooling and in any editor. 12 | 13 | For the Node package management, we decided to go with `pnpm` rather than vanilla `npm` or `yarn`. The main motivation behind `pnpm` is one of the challenges of this project: saving space and time by reusing Node packages across potentially many sketches, not least the required `electron` and `p5`. `pnpm` also doubles as a Node version manager and has proven to be a promising tool on the JavaScript side of this project. More information can be found in [the project’s wiki](https://github.com/stephanmax/processing4/wiki/Chosen-Technologies#node-package-management) where I document the reasoning behind chosen technologies. 14 | 15 | ## Challenges and Solutions 16 | 17 | The big challenge of this milestone was getting into user interface programming with Kotlin and [Compose](https://www.jetbrains.com/help/kotlin-multiplatform-dev/compose-multiplatform.html). I am used to building UI for the web with constant access to a browser’s developer tools and years of experience in established patterns that tie together UI, state, and behavior. Thankfully, online resources and communities on Compose, its interoperability with Java’s Swing framework, and managing state are abundant and high-quality. I am enjoying coding in Kotlin more and more. 18 | 19 | ## Next Steps 20 | 21 | Month and milestone 3 are all about improving the mode’s UI and user experience and getting a working version in front of as many people as possible. That includes more editor capabilities (error reporting, autocomplete, syntax highlighting) and a better communication to the user what processes are running behind the scenes and when they are finished; think Node package installation, what packages are installed, hints that the HTML file that is loaded in Electron needs updating when sketch files are renamed or added… or doing that automatically even. 22 | 23 | We also decided to add a compelling example sketch in TypeScript to demonstrate how JavaScript build tooling and Processing can cooperate further. 24 | 25 | ## Relevant Resources 26 | 27 | - [Project repository](https://github.com/stephanmax/processing4/tree/main) with the [working branch](https://github.com/stephanmax/processing4/tree/pr05-poc) 28 | - [Project wiki](https://github.com/stephanmax/processing4/wiki) 29 | 30 | ### Recommended Learning 31 | 32 | - [Programming Kotlin](https://pragprog.com/titles/vskotlin/programming-kotlin/) by Venkat Subramaniam (Especially chapters on Lambdas and Coroutines) 33 | - [Philipp Lackner’s Youtube channel](https://www.youtube.com/@PhilippLackner/videos) has been a great source for introductory videos around Kotlin and Compose for me. I appreciate a short bird’s eye view on things before diving into the docs and manuals. 34 | -------------------------------------------------------------------------------- /2025_BuildingBridges/monthly-reports/pr05_2025_September_report_Stephan_Max.md: -------------------------------------------------------------------------------- 1 | # Monthly Report – September: Desktop Support for p5.js via a New Processing Mode 2 | 3 | ## Overview 4 | 5 | The goal of this month was to improve the editor experience of the p5.js mode and make it more robust and stable. This included proper syntax highlighting for p5.js keywords, error reporting inside the Processing Development Environment, and auto-installing all the necessary tools and dependencies—pnpm, Node, and Node packages—to get a p5.js sketch running inside Processing. The latter is especially important, as it is a cornerstone of Processing’s accessibility that we do not assume the presence of any software on the user’s machine. Processing—and any mode, for that matter—should be batteries-included. 6 | 7 | ## Key Decisions and Progress 8 | 9 | ### Syntax Highlighting 10 | 11 | I generated keywords for p5.js based on the official reference JSON and for JavaScript based on Codemirror. That way, we stay as close to the implementation of syntax highlighting of the p5.js web editor as possible and hopefully cause no confusion for people that work in both environments. However, the current implementation of syntax highlighting via KEYWORDS.txt has its issues: It does not seem to be context-sensitive, so highlighting the property `.width` and the top-level variable `width` differently is not possible. That also leads to the wrong highlighting of the math function `log()` inside `console.log()`. It is also not clear what a JavaScript keyword is and how far down the official ECMAScript spec one wants to go. After a chat with my mentor Stef and considering the ongoing efforts to move Processing to a more robust LSP-based solution, we decided to not invest more time into a better solution for now. You can read more about how I generated the keywords in the [wiki](https://github.com/stephanmax/processing4/wiki/Syntax-Highlighting). 12 | 13 | ### Error Reporting 14 | 15 | Sketch errors happen inside the renderer processes of Electron where they are caught by implementing listeners for `error` (`throw`n errors) and `unhandledrejection` (rejected/unfulfilled `Promise`s) events. p5.js uses the same logic to implement its [Friendly Error System](https://p5js.org/contribute/friendly_error_system/). Errors are parsed and sent to the main process of Electron where they are printed to standard output and recorded by the PDE. I use `SketchException`s to report the error in PDE’s status bar and highlight the error inside the code editor. 16 | 17 | ### Auto-Install 18 | 19 | This month I started to add [releases](https://github.com/stephanmax/processing4/releases) as `.pdex` files. Double-clicking will install the mode inside Processing. Once the user restarts Processing, the mode will check for `pnpm` and `node` and install both if not found. Once all Node dependencies are installed, the user can go ahead with working on their sketch. This has been tested with Processing 4.4.7 on MacOS Monterey 12.7.6. 20 | 21 | ## Challenges and Solutions 22 | 23 | This month has been scattered with personal and professional challenges and I want to thank both Stef and Raphaël for providing me with the necessary order and support to help me get this mode into a serviceable state. 24 | 25 | The biggest challenge of this milestone was the auto-install, as it not only required me to test the automated runs of command line processes on different platforms, but also to restore an environment without `pnpm` and `node` after every test to guarantee that the whole setup and install process works according to plan. 26 | 27 | ## Next Steps 28 | 29 | Month and milestone 4 are about improving the mode’s stability further, adding more examples (most notably a TypeScript example to showcase more complex JavaScript tooling), adding a stand-alone app export feature, and releasing the mode as an official contribution for Processing’s contribution manager to install. I will keep the [p5.js mode README](https://github.com/stephanmax/processing4/blob/pr05-poc/p5js/README.md) updated with current features and remaining todos. 30 | 31 | ## Relevant Resources 32 | 33 | - [Project repository](https://github.com/stephanmax/processing4/tree/main) with the [working branch](https://github.com/stephanmax/processing4/tree/pr05-poc) 34 | - [README](https://github.com/stephanmax/processing4/blob/pr05-poc/p5js/README.md) 35 | - [Project wiki](https://github.com/stephanmax/processing4/wiki) 36 | 37 | ### Recommended Learning 38 | 39 | - [Kotlin From Scratch](https://nostarch.com/kotlin-scratch) by Faisal Islam is a project-based, playful, creative, yet surprisingly thorough introduction to Kotlin. 40 | -------------------------------------------------------------------------------- /2024_NewBeginnings/monthly-reports/pr05_2024_August_report_Diya_Solanki.md: -------------------------------------------------------------------------------- 1 | # August Monthly Report: Develop a VSCode LSP Server Extension for Processing 2 | By **Diya Solanki** 3 | 4 | - [Project Repository: Processing Language Server Extension](https://github.com/diyaayay/processing-language-server-extension/) 5 | - [Working Branch: main](https://github.com/diyaayay/processing-language-server-extension/blob/main) 6 | 7 | ## Completed Tasks (what's done so far) 8 | 9 | - [x] [Using the VSCode APIs](https://code.visualstudio.com/api/language-extensions/language-server-extension-guide) to set up a robust client-server architecture for establishing the connection between the server and the extension-host. 10 | - [x] Setup an `extension.ts` file which handles the spawning of the processing process, has the integrated `Run and Stop` buttons, processing commands for VSCode launch tasks, created a buffered channel, etc. 11 | - [x] [Resolved path](https://github.com/diyaayay/processing-language-server-extension/commit/dd887c14ef74cdba875abd59ecd210caf49b7c59) using `vscode-uri` which was earlier `filePathToURI` from `node:uri`. 12 | - [x] Researched into establishing an abstract syntax tree for processing language syntax; still wondering over ANTLR, lexer and parser; or writing a static AST. 13 | - [x] [Added Java Grammar support:](https://github.com/diyaayay/processing-language-server-extension/commit/7c91e7a99172740fda46ff796ea35af1a713cb55) Integrated the top level Java keywords like `private`, `static`; class body keywords like `extends`; method body keywords like `syscalls(), trim()` etc. 14 | - [x] [Added standard keyword support for processing](https://github.com/diyaayay/processing-language-server-extension/commit/74eb1cb54fef02381c3be6542b73b6c665425f3a) which includes the code snippets for Java processing. 15 | - [x] Analysed a Processing [VSCode Extension](https://github.com/avinZarlez/processing-vscode) by Avin to see the approach of running processing sketches inside the extension host. `Conclusion as of now`: We are using the similar approach as of now, i.e., to use the `child_process` library to spawn a processing process; this is a work in progress. 16 | - [x] `Solved`: Other features such as `Run & Stop` buttons (for running and stopping sketches from rendering) in the extension host for ease of use has been integrated in the client-side (can be found in the extension host). 17 | 18 | 19 | ## Challenges Encountered 20 | 21 | - Currently, I have created some [symlinks](https://www.linode.com/docs/guides/linux-symlinks) which : a) points to some [inodes](https://docs.rackspace.com/docs/what-are-inodes-in-linux), and b) references to some of the dependencies that are required to run the processing binary. So, as of now, the sketch renders (inside the extension host) but it requires some setup. The main challenge is to make the version migration work everywhere - which I am currently working on. 22 | - We aim to run the processing sketches inside the extension host (which is different from running the sketches in a JavaScript frontend, for example). We could simply call the binary using some `npm` package like `child_process`, `spawn-process`, etc., but then we need it inside the extension host as it should also have the client-server support - necessary to make all LSP features work smoothly. 23 | - I am reading a bit more on symlinks and exploring what all options do we have for the wrappers. For example, [vscode-languageserver-node](https://github.com/microsoft/vscode-languageserver-node) is using a custom script called `symlink` which can be executed normally as `npm run symlink`. It just executes a file called [symlink.js](https://github.com/microsoft/vscode-languageserver-node/blob/main/build/bin/symlink.js) which creates the hardlinks and softlinks between the protocol, the server, the client and the extension folder. Some languages have better low-level control, like [C](https://pubs.opengroup.org/onlinepubs/009696899/functions/symlink.html#:~:text=The%20symlink()%20function%20shall,contained%20in%20the%20symbolic%20link) and [Rust](https://doc.rust-lang.org/stable/std/os/unix/fs/fn.symlink.html), if that can help better with the system level symlinks; I am exploring these options also. 24 | - After this is done, it will be ready for testing and releasing, ASAP. 25 | 26 | ## Plans for September 27 | 28 | - [ ] `Goal`: Two features per week: 29 | - [ ] Code-completion 30 | - [ ] Diagonostics 31 | - [ ] Hover 32 | - [ ] Syntax highlighting 33 | - [ ] `onReferences` 34 | - [ ] `onDefinition` 35 | - [ ] `...` 36 | 37 | - [ ] Implement the fully-featured set LSP. 38 | - [ ] Export the extension into a `.vsix` file and publish it to the [VSCode Market Place](https://marketplace.visualstudio.com/vscode). 39 | - [ ] Detailed documentation for installing and configuring the extension. -------------------------------------------------------------------------------- /2024_NewBeginnings/monthly-reports/pr05_2024_September_report_Diya_Solanki.md: -------------------------------------------------------------------------------- 1 | # September Monthly Report: Develop a VSCode LSP Server Extension for Processing 2 | By **Diya Solanki** 3 | 4 | - [Project Repository: Processing Language Server Extension](https://github.com/diyaayay/processing-language-server-extension/) 5 | - [Working Branch: main](https://github.com/diyaayay/processing-language-server-extension/tree/main) 6 | - [Testing Branch: test](https://github.com/diyaayay/processing-language-server-extension/tree/test) 7 | 8 | ## Completed Tasks (what's done so far in September) 9 | 10 | - [x] [Test Branch Setup](https://github.com/diyaayay/processing-language-server-extension/tree/test) A test branch was created to install and run the extension on any platform, including the ability to run Processing sketches on Windows, along with LSP features. Clone the test branch to try out the Processing LSP Extension. 11 | - [x] [Project Setup Simplification](https://github.com/diyaayay/processing-language-server-extension/tree/test?tab=readme-ov-file#installation) Simplified the project setup process to use minimal commands, improving accessibility for users. 12 | - [x] [Project Dependencies release](https://github.com/diyaayay/processing-language-server-extension/releases/tag/v1.0) Downloads the Processing sketch-running dependencies on setting up the extension locally. 13 | - [x] [Deps download script](https://github.com/diyaayay/processing-language-server-extension/blob/test/dependency.bat) A script was created to run before `npm install`, automating the download and extraction of dependencies. 14 | - [x] [On-hover Description](https://github.com/diyaayay/processing-language-server-extension/tree/test?tab=readme-ov-file#on-hover-description) Hovering the mouse over a keyword now displays a pop-up with the meaning of the keyword. 15 | - [x] [Go-to Definition](https://github.com/diyaayay/processing-language-server-extension/tree/test?tab=readme-ov-file#go-to-definition) The "Go to Definition" feature allows users to navigate to the definition of a symbol using cmd+click or by right-clicking the variable name and selecting `Go To Definition`. 16 | - [x] [Go-to References](https://github.com/diyaayay/processing-language-server-extension/tree/test?tab=readme-ov-file#go-to-references) The references request is sent from the client to the server to resolve project-wide references for the symbol denoted by the given text document position. This feature finds all the references of a variable, class or a function used. 17 | Users can right-click or click on the `n` References link that appears over declarations. 18 | - [x] [Code Completion](https://github.com/diyaayay/processing-language-server-extension/tree/test?tab=readme-ov-file#code-completion) Context-aware suggestions are provided to speed up coding. 19 | - [x] `Running Processing Sketches Headless`: The `ctrl+shift+p` command or the `Run` button in the top right corner runs Processing sketch in the user's system. 20 | 21 | 22 | ## Challenges Encountered 23 | - **Diagnostics** for Processing sketches from Processing’s sketch validation logic is still a work in progress. 24 | - **Running Processing headless** without requiring the user to set paths or require some complex setup was the main challenge to make the version migration work everywhere. 25 | - **Dependency Management:** Handling external dependencies through symlinks and scripts proved tricky. Ensuring that the right libraries were accessible, while maintaining compatibility with different system environments, added complexity. 26 | - This required careful handling of directory structures, ensuring each symlink pointed to the correct target without causing confusion or overlap. 27 | - **Error Handling in Symlink Creation:** Error handling for symlink creation was important to ensure that if a symlink already existed or failed to be created, the system handled it gracefully without halting the entire setup process. 28 | - After setting up the script to download and unzip the dependencies, integrating them in the `sketchRunner.ts`, and testing them locally, I set up a Windows VM to clone the test branch and ensure that the extension is working on other machines. 29 | 30 | 31 | ## Plans for October 32 | 33 | - [ ] `Goal`: Bug Fixes in: 34 | - [ ] Code-completion 35 | - [x] Hover (no known bugs) 36 | - [x] Syntax highlighting (no known bugs) 37 | - [ ] `onReferences` 38 | - [ ] `onDefinition` 39 | - [ ] `...` 40 | 41 | - [ ] Enhance the UI for sketch/sketchbook and running the sketches. 42 | - [ ] Export the extension into a `.vsix` file and publish it to the [VSCode Market Place](https://marketplace.visualstudio.com/vscode). 43 | - [ ] Detailed documentation for further development of the project by new contributors. 44 | - [ ] Detailed documentation for installing and configuring the extension. -------------------------------------------------------------------------------- /2024_NewBeginnings/monthly-reports/pr05_2024_September_report_Nahee_Kim.md: -------------------------------------------------------------------------------- 1 | # Editor Migration Progress Report - September 2024 2 | 3 | Throughout September, I've been working diligently to transition from the class-based React implementation using CodeMirror 5 to a more modern functional component approach with CodeMirror 6. This update aims to improve the editor's performance, maintainability, and align it with current best practices. 4 | 5 | ## 1. React Component Modernization 6 | 7 | - **Functional Component Conversion**: I've replaced the class-based structure with a functional component. This shift allows me to leverage React hooks, resulting in more concise and easier-to-understand code. 8 | 9 | - **State Management Overhaul**: By utilizing hooks like useState, useEffect, useRef, and useCallback, I've streamlined the state management process. This new approach provides better control over the component's lifecycle and state updates. 10 | 11 | - **Lifecycle Method Adaptation**: I've transitioned away from traditional lifecycle methods (such as componentDidMount and componentDidUpdate) to the more flexible useEffect hook. This change gives me finer-grained control over side effects and helps prevent some common bugs associated with class components. 12 | 13 | ## 2. CodeMirror 6 Integration 14 | 15 | - **Extension-Based Architecture**: CodeMirror 6 introduces a more modular, extension-driven system. I've adapted the codebase to this new paradigm, which allows for greater flexibility and easier customization of editor features. 16 | 17 | - **State and View Management**: I've implemented the new EditorState and EditorView concepts from CodeMirror 6. This gives me more granular control over the editor's behavior and rendering. 18 | 19 | - **Feature Reimplementation**: I've reestablished core functionalities like syntax highlighting, line numbers, and code folding using CodeMirror 6's extension system. This process required a deep dive into the new API but has resulted in a more robust implementation. 20 | 21 | ## 3. Editor Configuration and Customization 22 | 23 | - **Keymap Customization**: I've rebuilt the custom key bindings using CodeMirror 6's keymap extension. This includes shortcuts for actions like code formatting, find/replace operations, and indentation control. 24 | 25 | - **Prettier Integration**: I've updated the code formatting function to use Prettier's formatWithCursor method. This ensures that I maintain cursor position after formatting, which greatly improves the user experience when working with JavaScript, CSS, and HTML. 26 | 27 | - **Autocompletion Enhancements**: I've reimplemented the autocompletion system using a combination of Fuse.js for flexible matching and CodeMirror 6's autocompletion extension. This new setup provides more accurate and context-aware code suggestions. 28 | 29 | These customizations help maintain the familiar feel of the editor while leveraging the new capabilities of CodeMirror 6. 30 | 31 | ## 4. Runtime Error Visualization 32 | 33 | - **Error Parsing**: Using StackTrace.js, I now parse runtime errors to extract relevant information such as line numbers and error messages. 34 | 35 | - **Visual Error Highlighting**: Leveraging CodeMirror 6's Decoration API, I've implemented a system that visually highlights lines in the editor where runtime errors occur. This provides in-context feedback to users. 36 | 37 | - **Dynamic Update Mechanism**: I use CodeMirror 6's StateEffect system to manage these error decorations, ensuring that they update efficiently as the code changes or errors are resolved. 38 | 39 | This feature significantly improves the debugging process by providing immediate visual cues within the editing environment. 40 | 41 | ## 5. Challenges and Lessons Learned 42 | 43 | Throughout this migration, I've encountered and overcome several challenges: 44 | 45 | - Adapting to CodeMirror 6's more modular architecture required a shift in my thinking about how editor features are implemented and composed. 46 | - Ensuring feature parity with the previous implementation while leveraging new capabilities has been a balancing act. 47 | - Performance optimization, particularly for large files, has required careful consideration and testing. 48 | 49 | These challenges have provided valuable learning experiences and have ultimately led to a more robust and flexible editor implementation. 50 | 51 | ## 7. Next Steps 52 | 53 | As I move forward, my focus will be on: 54 | 55 | - Developing a custom React hook (useCodemirror) to encapsulate editor initialization logic, promoting code reuse and maintainability. 56 | - Conducting thorough performance testing and optimization, with a particular focus on handling large files and complex operations. 57 | - Further enhancing the error handling and debugging tools to provide an even better development experience. 58 | - Exploring additional CodeMirror 6 extensions that could bring new, valuable features to the editor. 59 | 60 | ## Conclusion 61 | 62 | The migration to a functional component using CodeMirror 6 represents a step forward for the editor. While there's still work to be done, the foundation I've laid this month sets us up for improved performance, better maintainability, and enhanced features in the future. 63 | 64 | Thank you for your ongoing support and collaboration on this project. 65 | -------------------------------------------------------------------------------- /2024_NewBeginnings/final-reports/pr05_2024_final_report_Miaoye_Que.md: -------------------------------------------------------------------------------- 1 | 2 | # Revamp the Friendly Error System in p5.js 2.0 3 | 4 | | **Project** | [Revamp the Friendly Error System in p5.js 2.0](https://github.com/processing/pr05-grant/wiki/Revamp-the-Friendly-Error-System-(FES)-for-p5.js-2.0-(detailed-description)) | 5 | | :--- | :--- | 6 | | **Grantee** | [Miaoye Que](https://github.com/sproutleaf) | 7 | | **Mentors** | [Kenneth Lim](https://github.com/limzykenneth), [Dave Pagurek](https://github.com/davepagurek) | 8 | | **Repo**| https://github.com/processing/p5.js/tree/dev-2.0 | 9 | 10 | ## Technical Decisions 11 | 12 | ### Condensing `parameterData.json` for Parameter Validation 13 | 14 | - The existing `parameterData.json` file that we use to extract parameter information has a few problems: 15 | - It often contains unnecessary information, such as the function's return type. 16 | - It contains information that can be represented in a more concise manner, `?` at the end of the parameter name to indicate that the parameter is optional. 17 | - Has a nested structure that makes it difficult to parse. 18 | - The improved `parameterData.json` file, generated by `utils/convert.js`, resolves the above issues and saw a reduction of 80% in file size. 19 | 20 | ### [Zod](https://zod.dev/) for Parameter Validation 21 | 22 | - Zod is a powerful, flexible schema validation library withs static type inference. It provides all the use cases that parameter validation in p5.js requires. 23 | - Zod is [already used in the p5.js ecosystem](https://github.com/processing/p5.js-website/blob/main/src/content/tutorials/config.ts#L19)—in the process of building p5's website. 24 | - Zod is tree-shakable, meaning that unused library code will not be included in the final p5 bundle size. 25 | - **Do note that** initially, we considered generating `.d.ts` files from the library code, then using the generated files as input in `ts-to-zod`, which will conveniently generate Zod schemas that can be readily used for parameter validation. However, this approach presented a few challenges, including: 26 | - The generated `.d.ts` files were too big and had duplicated information. 27 | - Chainable methods return an interface for the whole p5 module, instead of just referencing the existing p5 type. 28 | - Even with the `.d.ts` files generated, `ts-to-zod` [might not support all the JSDoc tags](https://github.com/processing/p5.js/pull/6972). 29 | - As a result, we wrote custom logic that parses the `parameterData.json` file containing all the function signatures for parameter validation. 30 | - We also experimented with using [`zod-validation-error`](https://github.com/causaly/zod-validation-error) to generate error messages, but eventually did not move forward with it because the library has a narrow range of customization options, limiting our ability to present error messages that are accessible, informative, and beginner-friendly. 31 | 32 | ### `acorn` and `acorn-walk` for detecting accidentally overriden constants and functions 33 | 34 | - We evaluated multiple JavaScript parsers, including `acorn`, `esprima`, and `espree`, and ultimately decided to use `acorn` and `acorn-walk` because: 35 | - `acorn` has a smaller bundle size and is faster than `esprima`. 36 | - `acorn` has better support for ES2015+ syntax. 37 | - `acorn-walk` has a simpler API and is easier to understand and modify. 38 | - Both `acorn` and `acorn-walk` are tree-shakable. 39 | 40 | ## Challenges 41 | 42 | ### Achieving a balance between leveraging on external libraries and keeping FES well-built and accessible 43 | 44 | - `ts-to-zod` and `zod-validation-error` were both great libraries that we considered using, but had to give up after some exploration due to their incompatibility with our project goals. 45 | 46 | ### Developing FES within the new p5.js 2.0 architecture 47 | 48 | - Many new facets of the p5.js 2.0 architecture (dependency injection, decorator pattern, etc) were unfamiliar to me, and some trial-and-error was needed to understand how to implement the 2 priorities in the project properly and efficiently, which delayed the initial project timeline. 49 | 50 | ### Ensuring comprehensive coverage of use cases 51 | 52 | - Both parameter validation and accidental override detection have many use and edge cases. Working on the development of these features required a detail-oriented attitude, and a robust understanding of FES. I made sure to document all the things to note, and worked through them diligently. 53 | 54 | ## Tasks Completed 55 | 56 | - [x] Condensed `parameterData.json` for parameter validation 57 | - [x] Implemented parameter validation with Zod 58 | - [x] Implemented accidental override detection 59 | - [x] Wrote unit tests for both parameter validation and accidental override detection 60 | 61 | ## Limitations 62 | 63 | As of now, the two features have not been integrated into the dev-2.0 branch in p5's repo yet, meaning that old versions are still being used. There also needs to be more testing in real environments (p5.js editor, processing, etc) to ensure that they work as expected in real-world scenarios. 64 | 65 | ## Work Remaining & Next Steps 66 | 67 | - [ ] Address the comments in the [last PR](https://github.com/processing/p5.js/pull/7326), and merge it. 68 | - [ ] Add more robust loading of user scripts. 69 | - [ ] Use the decorator pattern to integrate the two features into p5.js 2.0. 70 | - [ ] Clean up old code that's no longer being used. 71 | - [ ] Go through all open issues in p5 repo related to FES, and respond to them if this project can address them. 72 | 73 | ## Relevant PRs and Issues 74 | 75 | Please refer to the two master issues for all PRs I've created, other issues that mention parameter validation / accidental override detection, and additional notes and to-do lists: 76 | 77 | - [Parameter Validation](https://github.com/processing/p5.js/issues/7178) 78 | - [Accidental Override Detection](https://github.com/processing/p5.js/issues/7269) -------------------------------------------------------------------------------- /2025_BuildingBridges/monthly-reports/pr05_2025_September_report_Vaivaswat_Dubey.md: -------------------------------------------------------------------------------- 1 | # September Monthly Report: Visual Regression Testing Implementation 2 | 3 | By **Vaivaswat Dubey** 4 | 5 | [Current Implementation](https://github.com/Vaivaswat2244/processing4/tree/pr05-visualtests/visual-tests) 6 | 7 | ## System Architecture Progress 8 | Following the custom Kotlin implementation of the comparison algorithm in August, the focus for September was on building a complete, integrated visual regression testing framework for the Processing environment. The architecture now encpsulates sketch execution, image comparison and gives result. 9 | 10 | The core components now include: 11 | 12 | - VisualTestRunner: The central orchestration class that manages the entire test lifecycle, from capturing a sketch to comparing it against a baseline image and generating a final test result. 13 | - ImageComparator: The custom, self-contained pixel matching engine that uses a pixelmatch-inspired algorithm implemented entirely in Java, eliminating external dependencies. 14 | - SketchRunner: A dedicated PApplet instance that manages the execution of a user's sketch in an off-screen buffer, ensuring consistent, repeatable rendering for testing purposes. 15 | - ProcessingTestSuite: A class for organizing and running multiple tests, simplifying the management of large test suites. 16 | - BaselineManager and TestExecutor: Utility classes to streamline baseline creation and single-test execution, making it easier for developers to manage their visual tests from the command line. 17 | 18 | ## Work Accomplished 19 | ### Processing Testing Framework Implementation 20 | 21 | Successfully developed and integrated a comprehensive testing framework that allows developers to write and run visual tests for their Processing sketches with minimal boilerplate. This framework leverages all the components planned in the architectural phase and provides a complete solution from sketch to final report. 22 | 23 | Technical Implementation Details: 24 | 25 | - Sketch Isolation: The SketchRunner class provides a controlled environment for executing user sketches. It runs each sketch as its own PApplet instance, captures the final rendered image, and ensures consistent size() and noLoop() settings, which are critical for reproducible visual output. 26 | - Test Orchestration: The VisualTestRunner class was completed to handle the full testing workflow. It now correctly identifies first-time runs to create a baseline image, loads existing baselines for comparison, and, if a test fails, automatically saves a diff image for debugging purposes. This simplifies the developer's debugging process. 27 | - Pixel-by-Pixel Comparison: The custom ImageComparator now performs a thorough pixel comparison, using an algorithm that is a functional equivalent of the pixelmatch library. It correctly identifies differences, post-processes them to find clusters of changes, and can intelligently ignore minor, isolated pixel differences or "line shifts" that might occur due to rendering subtleties. 28 | - Simple Test Class: To check the workability of the SketchRunner, whipped up a class containing some basic tests such as different coloured squares and a gradient sketch. In future this will be replaced by `JUnit`. Similar workarounds will be made for the suite execution which is supposed to group selected tests together and then run them as a suite. 29 | 30 | ### Integration Testing and Validation 31 | 32 | Extensive testing was performed using a some simple tests, including shapes, gradients. This validation process confirmed that the framework correctly: 33 | 34 | - Creates Baselines: The system successfully creates __screenshots__ directories and saves new baselines on the first run. 35 | - Detects Changes: A modified sketch (e.g., changing a red circle to a green one) correctly fails the test and generates a diff image. 36 | - Handles Identical Runs: Running the same sketch multiple times with an existing baseline correctly results in a "PASSED" status. 37 | - Manages Minor Differences: The algorithm's ability to identify and filter out insignificant "line-like" clusters was validated, ensuring that minor anti-aliasing artifacts do not cause false failures. 38 | 39 | ## Challenges Encountered 40 | The primary challenge in September was managing the complexities of the Processing runtime itself. PApplet is not designed to be run non-interactively, so controlling its lifecycle and ensuring that it renders a single frame before exiting required careful use of noLoop(), runSketch(), and a robust waiting mechanism. The thread-based nature of PApplet.runSketch() required a separate synchronization mechanism to ensure the main test thread does not proceed until the sketch has fully rendered its output, which was a key aspect of the SketchRunner implementation. 41 | 42 | Additionally, ensuring the custom ImageComparator was robust enough to handle the variety of visual output, from simple shapes to complex, randomized patterns, required careful tuning of the clustering and lineShift detection logic to prevent false positives while still catching real regressions. 43 | 44 | ## Key Insights 45 | - Headless Processing is a non-trivial problem: The PApplet class is inherently tied to a visual window. Running it in a "headless" or non-interactive mode for testing purposes required a bespoke solution to manage the application lifecycle and capture the rendered output. 46 | - Custom pixel matching provides full control: The decision to implement a custom, Kotlin-based comparison algorithm has paid off by providing a completely self-contained solution. It avoids all the complexities and trade-offs of the binary investigation from August and gives us granular control over every aspect of the comparison logic. 47 | 48 | ## October Plans 49 | October will focus on polishing the existing implementation and preparing for a public release. 50 | 51 | ### Documentation and Examples: 52 | 53 | - Using JUnit rather than the independent classes being used right now. 54 | - Porting the visual tests currently in p5.js for Processing. 55 | - Write clear, easy-to-follow documentation and tutorials on how to set up and use the framework with a standard Processing sketch. 56 | - Develop a variety of example sketches that demonstrate different features of the tool, including how to handle different test configurations. 57 | 58 | The primary deliverable for October will be a public, well-documented, and stable version of the visual regression testing library for Processing, ready for use by the community. -------------------------------------------------------------------------------- /2025_BuildingBridges/monthly-reports/pr05_2025_July_report_Claire_Peng.md: -------------------------------------------------------------------------------- 1 | # Monthly Report - July: Incremental Typescript Migration for the p5.js Web Editor 2 | 3 | ## Overview 4 | 5 | The p5.js web editor is a legacy codebase running on Node 16, with over 100,000 lines of code. Given the limited time available for this grant, my goal was not to migrate the entire client or server folder at once, but to **prepare both parts of the codebase and remove friction so that future contributors could treat TypeScript migration tasks as approachable, “good first issues.”** 6 | 7 | To that end, I focused on configuring all necessary TypeScript tooling for both the server and client, and migrating as many files as possible within the rest of timeframe to serve as examples. 8 | 9 | In July, I completed the initial setup of the client folder for TypeScript migration, including installing dependencies and configuring the tooling. I also migrated instances of both non-React files and React components to validate that the setup worked across different types of client-side files. 10 | 11 | ## Key Decisions and Progress: 12 | 13 | - Setting up a root `tsconfig.json` that references individual configurations for both the client and server. 14 | - This allowed for incremental migration of the client and server separately. 15 | - Setting up a root `tsconfig.base.json` that the `client/tsconfig.json` and `server/tsconfig.json` can extend on. 16 | - Installed all Typescript dependencies. 17 | - Setting up a typecheck script to run as a pre-commit hook. 18 | - Configured TypeScript to automatically resolve imports without requiring explicit `.ts` or `.js` extensions. 19 | - Migrated an instance of a non-React file (`client/utils/generateRandomName`) 20 | - Migrated an instance of a React file (`client/common/SkipLink`) 21 | - Migrating the entire `client/utils` folder 22 | - Migrating the entire `client/common` folder 23 | 24 | ## Challenges and Solutions: 25 | 26 | The primary challenge was setting up the correct versions of dependencies that were compatible with Node 16. Many TypeScript-related packages required manual version pinning, as newer versions often conflicted with the legacy environment. 27 | 28 | Another challenge involved Prettier and ESLint integration. I encountered a bug where ESLint would indicate that Prettier flagged valid TypeScript syntax, such as the `as` keyword or angle brackets (``), as errors. I had already configured the ESLint setup to use the Babel parser for `.js` and `.jsx` files, and the ESLint TypeScript parser for `.ts` and `.tsx` files, so it was strange that this was happening. 29 | 30 | At first, I thought this may be due to us having older version of Prettier or ESLint, which may have come before these pieces of Typescript syntax were introduced. I then spent time trying to upgrade them and even experimented with bumping Node from 16 to 18, but this caused widespread peer dependency conflicts, so I reverted the changes. 31 | 32 | Eventually, I realized that the issue stemmed from Prettier being hardcoded to always use the Babel parser, which does not support TypeScript syntax, instead of different parsers depending on file-type as specified in the ESLint config. [Once I removed that hard-coded setting](https://github.com/processing/p5.js-web-editor/pull/3565/files#diff-663ade211b3a1552162de21c4031fcd16be99407aae5ceecbb491a2efc43d5d2L8), the TypeScript-related formatting errors disappeared. 33 | 34 | Another challenge was that in my initial migration work, I forgot to use `git mv someFile.js someFile.ts` when changing the file extension, which broke history tracking and required remaking the branch to preserve it. I ended up finding that this flow of copying over changes to a final branch quite useful for my working style. I could work out the tricky aspects of migration freely in a messy working branch, then create a final branch with a clean, legible and linear commit history that future open-source contributors would find easier to read. 35 | 36 | As for the actual migration process, it was mostly straightforward. I reviewed where each file or component was used, inferred the types of inputs and expected outputs, and added type annotations accordingly. In some cases, such as with the `getConfig()` function, I discovered issues with how edge cases were handled. That function retrieves environment variable values, but it sometimes returned `undefined`, which would be coerced to a `string` when used in template literals. Additionally, it sometimes returned `string` values when `booleans` or `numbers` were expected, which required explicit parsing, so I added a type-parsing utility function that can be used in conjunction with `getConfig()` in this instance. **For more details, please see PR #3 linked below.** 37 | 38 | I also found a few functions that were no longer being used anywhere and was able to remove them. Others were refactored for better readability. Whenever I made changes like these, I added unit tests prior to secure against regression — especially for code that was significantly altered during migration. 39 | 40 | ## PR's: 41 | 42 | - [pr05 Typescript #1: Set up TS dependencies (#3533)](https://github.com/processing/p5.js-web-editor/pull/3533) 43 | - Install ts dependencies for Node 16 and React 16 44 | - Set up tsconfigs & add typecheck on pre-commit hook 45 | - [pr05 Typescript #2: automatically resolve imports of ts files & migrate instance of client/utils file (#3540)](https://github.com/processing/p5.js-web-editor/pull/3540) 46 | - Auto-resolve imports of ts files 47 | - Migrate `client/utils/generateRandomName()` 48 | - [pr05 Typescript Migration 3: Migrate the client/utils folder (#3553)](https://github.com/processing/p5.js-web-editor/pull/3553) 49 | - Migrate entire client folder, with exception of CodeMirror & intellisense modules due to unfamiliarity 50 | - [pr05 Typescript #4: migrate instance of react file (#3554)](https://github.com/processing/p5.js-web-editor/pull/3554) 51 | - Install missing ts react dependencies 52 | - [pr05 Typescript Migration #5: Migrate client/common folder (#3565)](https://github.com/processing/p5.js-web-editor/pull/3565) 53 | - Fix issue with hard-coded `parser: babel` option in `.prettierrc` 54 | - Migrate entire `client/common` folder 55 | 56 | ## Next Steps: 57 | 58 | In August, I plan to continue migrating as much of the client folder as possible. My top priority is to migrate a Redux reducer as an example, since that area of the codebase is less familiar to me. I’ll start by studying existing reducer implementations and proceed from there. 59 | As well in August, I will start drafting documentation on how to migrate React files & how to write tests. 60 | -------------------------------------------------------------------------------- /2025_BuildingBridges/final-reports/pr05_2025_final_report_Stephan_Max.md: -------------------------------------------------------------------------------- 1 | # Final Report: Desktop Support for p5.js via a New Processing Mode 2 | 3 | ## Overview 4 | 5 | - Project 6 | - [Description](https://github.com/processing/pr05-grant/wiki/2025-pr05-Project-List#desktop-support-for-p5js-via-a-new-processing-mode) 7 | - Period: July–October 2025 8 | - People 9 | - Grantee: [Stephan Max](https://stephanmax.com) 10 | - Mentor: [Stef Tervelde](https://github.com/Stefterv) 11 | - Links 12 | - [Repository](https://github.com/stephanmax/processing-p5.js-mode) 13 | - [Wiki](https://github.com/stephanmax/processing-p5.js-mode/wiki) 14 | - Links point towards by personal GitHub page. The mode *will* move to the official Processing organization on GitHub and GitHub maintains redirects whenever repository ownership is transferred, so there should be now disruptions. 15 | 16 | ## Technical Decisions 17 | 18 | ### General 19 | 20 | This mode has been developed in Kotlin and all mode-specific implementations can be found in the top-level [p5js](https://github.com/stephanmax/processing-p5.js-mode/tree/pr05-poc/p5js) folder. Necessary build steps (general build, generate `pdex` file for distribution, in the future: download p5.js examples) have been specified with Gradle. Hence, this mode complies with the general implementation direction of Processing. 21 | 22 | We decided to go with Electron as our embedded browser as it is a mature product with a big user base and excellent documentation. We are aware that there are competing products out there, most notably Tauri, and we will keep on monitoring them as we move forward with this mode. 23 | 24 | ### Node Package Management 25 | 26 | We chose [PNPM](https://pnpm.io) as the underlying node package manager. PNPM stores dependencies and version diffs in a central store and allows dependencies across sketches, for example Electron, to be reused. That way we save a lot of space and installation time. You can read more about the decision process [in the wiki](https://github.com/stephanmax/processing-p5.js-mode/wiki/Chosen-Technologies). 27 | 28 | Bonus: We realized that `pnpm` can also be used as a Node version manager which solves a second issue for us. `pnpm` is therefore our one-stop-shop for all things Node required by the mode. 29 | 30 | ### Packaging Distributable Apps 31 | 32 | We evaluated both [Electron Forge](https://www.electronforge.io) and [electron-builder](https://www.electron.build/index.html) for turning p5.js sketches into distributable apps. Although the former seems to be the officially endorsed solution by Electron, its configuration has been proven to be complicated and interop with `pnpm` at least complicated. `electron-builder` has worked almost out-of-the-box and we managed to export sketch apps for Linux, MacOS, and Windows. 33 | 34 | ### Distribution of the Mode 35 | 36 | We created a Gradle task that builds the mode into installable `pdex` files that have been distributed as [beta relases on GitHub](https://github.com/stephanmax/processing-p5.js-mode/releases). By the end of November, this mode will appear in the official [contributions manager](https://github.com/processing/processing-contributions) of the Processing Development Environment. 37 | 38 | ## Challenges 39 | 40 | - A lof of new things to learn: Kotlin (especially concurrent programming, the mix of object-oriented and functional programming, and Java interop), Gradle (in Kotlin), Compose Multiplatform for UI, and some Electron intricacies. 41 | - Reversing my usual modus operandi—getting all the requirements first and then execute the implementation—to getting a product, that we as developers think will be helpful and meaningful, in front of the actual users as soon as possible and derive the actual requirements from them. That was a new spin for me and sometimes collided with my general tendency to drill deep into technical issues rather than work broadly to generate as much value as possible in a restricted timeframe. 42 | - The mode relies on a critical amount of moving parts to get everything working: `pnpm` for package and Node management, Node JavaScript runtime for basic inner workings, Electron to display sketches, `electron-builder` for exporting sketches as executables. Getting all these tools in place on all relevant operating systems and play nice with each other has been challenging. 43 | - Personal struggles impeded the timeline. 44 | 45 | All these challenges—especially the last one—have been met by Stef Tervelde and Raphaël de Courville with patience, understanding, and the necessary amount of project management support to push things over the finish line. 46 | 47 | ## Completed Tasks 48 | 49 | - [x] Editor Experience 50 | - [x] Communication between Electron and Processing via Electron’s inter-process and standard I/O process communication 51 | - [x] Baseline: Run/Stop buttons, sketch size as per `createCanvas()`, hide non-js and Electron boilerplate 52 | - [x] Syntax highlighting; we are aware that the underlying system for syntax highlighting has its flaws as is likely to change in the future. You can read more on syntax highlighting [in the wiki](https://github.com/stephanmax/processing-p5.js-mode/wiki/Syntax-Highlighting). 53 | - [x] Examples 54 | - [x] Error reporting 55 | - [x] Package installation via pnpm 56 | - [x] Auto-install all necessary dependencies (pnpm, Node, Electron) on Linux, MacOS, and Windows 57 | - [x] Packaging sketches as distributable apps for Linux, MacOS, Windows 58 | - [x] Proof-of-concept TypeScript example 59 | 60 | ## Limitations and Next Steps 61 | 62 | I would like to fix some unpleasant bugs in the remainder of November and have the mode moved to the Processing GitHub organization and into the contribution manager by the end of November. These bugs are mainly around general stability and addressing some error messages that we have encountered. [Issues are tracked via GitHub.](https://github.com/stephanmax/processing-p5.js-mode/issues) 63 | 64 | ### Other Work Remaining 65 | 66 | - [ ] Better TypeScript example; include all official p5.js examples 67 | - [ ] Quality-of-life improvements around app export (icon, app title, non-hardcoded export settings) 68 | - [ ] Button to open developer tools in Electron’s Chromium browser 69 | - [ ] Add JavaScript linting to error reporting and allow errors/warnings to show without running the sketch first 70 | - [ ] Remove package installation UI for now and rather add a menu action for user’s to edit `package.json` 71 | 72 | ## Closing Thoughts 73 | 74 | This grant was an exciting and empowering experience from start to finish. I felt the right amount of challenge outside of my comfort zone and appreciation of my talents. The level of professional and personal learning for me is unprecedented. I am grateful for all the people involved and for being a part of this warm-hearted community. Thank you, Processing Foundation! 75 | -------------------------------------------------------------------------------- /2024_NewBeginnings/final-reports/pr05_2024_Final_report_DoraDo.md: -------------------------------------------------------------------------------- 1 | # Prototype a Collaborative Editor for Processing - Final Report 2 | ## Technical Decisions 3 | Throughout this project, several technical choices were crucial in achieving a stable, cross-platform application. Here are the primary decisions: 4 | 5 | - UI Design & Development: 6 | - React & Zustand: React was chosen for its component-based architecture and Zustand for state management. This combination provided a scalable and maintainable structure for the UI. 7 | - Radix UI Component Library: Integrated Radix UI components to enhance the app's visual design and user experience. This library provided a consistent design system for future contributors and improved the overall UI. 8 | - Codemirror: Originally, we considered Theia, but opted for CodeMirror due to its ease of implementation, extensive use in the p5 Web Editor, existing collaborative features, and faster development potential. This choice was validated by the successful integration of CodeMirror with Electron. 9 | 10 | - Real-time Collaboration: 11 | - CodeMirror & YJS Integration: We successfully integrated the YJS framework with the CodeMirror editor to enable real-time collaboration. This integration was essential for the core functionality of the app and laid the foundation for collaborative editing. We used the `y-codemirror.next` package with the `y-websocket` provider for real-time synchronization of CodeMirror documents. 12 | - Websockets: We used `y-websocket` for the WebSocket provider to enable real-time collaboration. This choice was made after evaluating WebRTC's limitations in handling larger groups effectively due to performance issues. We decided that Websockets were more stable for our use case and provided better support for scaling. Currently, the server is hosted on a cloud service called Render on their free-tier plan, which puts the server to sleep after a period of inactivity. The repository to the server can be found [here](https://github.com/doradocodes/pce-server). When a user hosts a collaborative session, a room is created on the server with a unique ID (the Room ID). This ID is then shared to the other users who want to join the session. Since the implementation was a bit crude for this prototype, rooms are not deleted after the session ends. But if a user drops from a session, they can still rejoin the session using the same Room ID, even if the host disconnects. 13 | 14 | - Running Processing Sketches: 15 | - Integrating Processing sketches was foundational to the app’s core functionality. In this prototype, sketches are saved locally as `.pde` files and Electron runs the `processing-java` executable to execute Processing sketches with a path to the file. It is not the most efficient method of running sketches, but it allowed us to demonstrate the collaborative editing of Processing code. Future iterations will include a headless version of Processing for more efficient sketch execution. 16 | 17 | - App Build and Distribution: 18 | - We initially tested app distribution using electron-builder, but challenges in handling Apple’s certificate verification led to adopting electron-forge, which streamlined the macOS notarization process. electron-forge provided more reliable handling for platform-specific requirements, making it easier to sign and notarize the app for Mac and set up for Windows. There was also an issue with notarizing the Processing library, which was resolved by creating a script called `sign_jnilib_files.js` to sign the library for notarization. 19 | - In order for others to build the app, contributors can use the provided `.env.example` file to set up their environment variables. 20 | 21 | 22 | ## Tasks Completed 23 | - Created a functional prototype of a collaborative code editor for Processing using Electron and CodeMirror. 24 | - Allow creation, editing, renaming, deleting, and running of local Processing sketches. 25 | - Integrated YJS with CodeMirror for real-time collaboration, enabling multiple users to edit a Processing sketch simultaneously. 26 | - Allows user selection of dark or light mode for the editor. 27 | - Builds for macOS and Windows. 28 | 29 | 30 | ## Challenges 31 | - Certificate and Notarization Issues: Encountering "damaged app" errors when testing macOS builds on other machines highlighted the need for proper app signing. After troubleshooting with Apple Developer tools and certificates, switching to electron-forge resolved this issue. 32 | 33 | - Cross-Platform Compatibility: Distribution on Windows and Linux introduced different runtime and dependency challenges. Upcoming tasks will focus on thoroughly testing builds on these platforms. 34 | 35 | - UI Bug Fixes: Addressing minor yet crucial UI bugs required repeated testing. Adding collapsible panels and submenus also introduced layout adjustments that required careful tuning to avoid disrupting the user flow. 36 | 37 | 38 | ## Limitations 39 | - Apple Developer Account Dependency: Open-source contributors must have Apple Developer credentials for macOS builds, which may limit participation without a centralized solution. 40 | - Websocket server: A crude websocket server was used for real-time collaboration, which may not be scalable for larger groups. Future iterations will explore more robust solutions for handling collaborative editing. 41 | - Processing Java library and JDK storage: For this project, the Processing Java library, nor the JDKm were committed to the repository due to its size. Contributors must download the Processing4 application, go into it's Contents folder and copy the Java folder into the tools/PlugIns folder. This is a temporary solution until a more automated process is implemented. 42 | - Windows build: The Windows build has not been fully developed or tested, since my development environment is macOS. Future work will focus on ensuring the app runs smoothly on Windows systems. 43 | - Single Tab View: The current version only supports a single tab for editing sketches since it was out of scope for the timeline of this prototype. 44 | - Separate Collaboration Window: In order to keep the Websocket connection straightforward, when the user starts a collaborative session, a new window is opened with the collaborative session. This was a solution to make connection/disconnection to specific rooms easier, since connection to a room is by URL. A future iteration should maintain rooms via Websocket messages so that the user could have multiple collaborative sessions open at once on the same window. 45 | 46 | ## Future Vision & Improvements 47 | - Running sketches with a headless Processing Library: This will streamline the execution of Processing sketches within the app, improving performance and user experience. 48 | - LSP Integration: Implementing the Processing Language Server Protocol will enhance coding features, such as autocompletion and syntax highlighting, for a more robust editing experience. 49 | - Allow Imports of External Processing Libraries: Enabling users to import external Processing libraries will expand the app's functionality and utility for Processing users. 50 | - Importing Assets: Adding support for importing assets like images and sounds into the Processing sketch will enhance the creative capabilities of users. 51 | - Peer-to-peer Collaboration: Exploring peer-to-peer connections with WebRTC for real-time collaboration will provide an alternative to WebSocket-based collaboration, offering more flexibility and scalability. 52 | -------------------------------------------------------------------------------- /2025_BuildingBridges/monthly-reports/pr05_2025_September_report_Claire_Peng.md: -------------------------------------------------------------------------------- 1 | # Monthly Report - September: Incremental Typescript Migration for the p5.js Web Editor 2 | 3 | ## Overview 4 | 5 | Over September, I pivoted from the previous work on migrating the `client` folder to concentrating on setting up migration for the `server` folder. This included: 6 | 7 | - Setting up TS dependencies on the `server` folder 8 | - Migrating the `server/routes` folder 9 | - Migrating server files related to emailing capabilities 10 | - Migrating the `User` model and `ApiKey` schema 11 | - Starting to migrate the `User` controller 12 | 13 | The decision to pivot from the `client` folder to the `server` folder came from [attempting to migrate an instance of redux reducers/actions](<(https://github.com/processing/p5.js-web-editor/pull/3633)>), where I chose the user preferences reducers/actions and I found that it would greatly improve developer experience by adding types to the server, so that it could act as a source-of-truth for API specs that could be found easily at a glance. 14 | 15 | I also addressed feedback from last month's PR's for the `client` folder. 16 | 17 | ## Challenges, Progress & Key Decisions: 18 | 19 | This month started with easy-wins and ended with some great challenges, as the first steps of `server` folder migration started with repeating some flows I had learned in the previous two months on the `client` folder migration; then ended with learning about the intricacies of `mongoose` Typescript syntax, which I am still continuing to learn about. 20 | 21 | ### Setting Up TS Dependencies on the `server` folder: 22 | 23 | Following what I learned for the client migration and had already set up for the repo, setting up TS dependencies on the `server` folder went relatively smoothly. I made the following changes: 24 | 25 | - Set up tsconfig for the `server` folder 26 | - Add a `typecheck:server` command to `package.json`, and manually checked that `typecheck:server` flags type-errors on .ts and .tsx files, and that it checks the `server` folder independently of the `client` folder 27 | 28 | I then tested my work by migrating `/server/utils/generateFileSystemSafeName`, chosen as it was a simple file to understand. Upon running the app locally, I found that there were additional configurations needed, as webpack and babel weren't able to automatically resolve files ending in `ts`. Resolving these required some research but were the steps below: 29 | 30 | - Updating `nodemon` to hot-reload upon ts-file updates 31 | - Updating `webpack` to resolve extensions with server folder modules 32 | - Updating `root/index.js` configurations for `@babel/register` to resolve server files with `@babel/preset-typescript` where appropriate. 33 | 34 | ### Migrating the email system: 35 | 36 | I initially intended to migrate the entire `server/views` folder, but I found that there were many files related to rendering email templates, so I thought it would be a good workflow to migrate the email system. This included: 37 | 38 | - Adding a `server/types` folder, with the first file being `email` to store all email-related types 39 | - All proceeding types will be stored in a barrel structure and exported to a `server/types/index.ts` 40 | - Migrating & extracting types from: 41 | - `server/views/mail.ts` 42 | - `server/views/consolidationMailLayout.ts` 43 | - `server/utils/renderMjml.ts` 44 | - `server/utils/mail.ts` 45 | - Updating any `default` exports to `named` exports, as per the style of the `client` files. 46 | 47 | ### Initial migration attempt for the User model, routes, controller: 48 | 49 | [I then attempted to migrate the entire `User` "system" (model, routes, controller)](https://github.com/clairep94/p5.js-web-editor/pull/13/files), but this proved extremely difficult, as I didn't know much about Mongoose Typescript best practices. I started migrating the controller file by adding types directly to the `req` and `res` parameters, but I ran into unexpected type errors in the routes file. It turned out that Express expects controllers to use the `RequestHandler` type, with the type parameters defined in the angle brackets to specify things like route params, response body, and request body. 50 | 51 | I also did not add unit tests during this initial attempt, so it felt risky as the codebase did not have existing tests for the User model, and only had tests for the API Key portion of the User controller. 52 | 53 | I felt it was best to re-start and take things much slower, focusing on the User routes, and User model for the remainder of the month, adding tests as I go to ensure that nothing regresses during migration. 54 | 55 | ### Migrating the Routes folder: 56 | 57 | After this pivot, I did a bulk migration of the `server/routes` folder, as I knew these files would not require many changes or need to extract types. One improvement from this work was that upon updating files to `ts`, I was hinted by the IDE to update some outdated syntax for creating a `Router` instance. 58 | 59 | 60 | 61 | While a super small change, a lot of files in the `server` folder have not been actively worked on for several years, and without typescript, we would not have known that this syntax was outdated which may have risked unexpected bugs. 62 | 63 | 64 | 65 | ### Migrating the User Model: 66 | 67 | I then worked on migrating `server/models/user`. As mentioned this file did not have corresponding unit tests for any of the model methods, so it was risky to migrate without first adding tests. I felt the code organisation could also be improved as the `server/models/user` file also included the `apiKey` schema, which could be moved to its own file for better separation of concerns. 68 | 69 | To this end, I: 70 | 71 | - Added unit tests for `apiKey` schema methods 72 | - Added unit tests for `user` model methods, and installed `mongodb-memory-server` which creates an in-memory test mongodb to run tests against locally and on CI 73 | - Migrated `apiKey` to its on file within `/server/models` 74 | - Extracted definitions for the `Document` and `Model` types for both `apiKey` and `user` 75 | 76 | I found the last step quite challenging, as I had never worked on mongoose with typescript before, and defining types can be quite verbose. 77 | 78 | The below is a best-effort at extracting types, which I believe does improve legibility for the codebase and give contributors a way to see definitions at a glance. 79 | 80 | 81 | 82 | 83 | 84 | 85 | 86 | 87 | 88 | 89 | 90 | 91 | 92 | For example, we now have a definition for `UserPreferences`, which can then be used as the source of truth for the `client` redux system for user preferences. 93 | 94 | ### Migrating the User Controller: 95 | 96 | For the remainder of the month, I worked towards migrating the user controller, which I will discuss more in detail in October's report upon completion of that file. 97 | 98 | ## PR's: 99 | 100 | ### Updated from previous month to address feedback & now merged: 101 | 102 | - [pr05 Typescript #8: migrate client/components/Menubar/MenubarSubmenu](https://github.com/processing/p5.js-web-editor/pull/3623) 103 | 104 | ### Current Month PR's - Merged: 105 | 106 | - [pr05 Typescript Migration #10: Setup Server TS Dependencies & migrate instance of server file](https://github.com/processing/p5.js-web-editor/pull/3636) 107 | - [pr05 Typescript Migration #11: Migrate server/routes folder](https://github.com/processing/p5.js-web-editor/pull/3643) 108 | - [pr05 Typescript Migration 12: Migrate server files related to emailing/nodemailer](https://github.com/processing/p5.js-web-editor/pull/3658) 109 | - [pr05 Typescript Migration #13: Migrate the User Model (& API Key schema)](https://github.com/processing/p5.js-web-editor/pull/3672/files) 110 | 111 | ## Next Steps: 112 | 113 | For the remaining time in October, I will focus on completing the User model, and if time allows, I will finish the user preferences redux. Having these two completed would mean that we have an instance of migrating each 'kind' of file on the repo, and enable new contributors to have example code / workflows that they can follow to migrate other portions of the codebase. 114 | 115 | During the quiet period, I will focus on writing documentation to explain how to migrate each file instance. 116 | -------------------------------------------------------------------------------- /2025_BuildingBridges/monthly-reports/pr05_2025_August_report_Vaivaswat_Dubey.md: -------------------------------------------------------------------------------- 1 | # August Monthly Report: Visual Regression Testing Implementation 2 | By **Vaivaswat Dubey** 3 | - [Project Repository: Processing Comparison Engine](https://github.com/Vaivaswat2244/visual-regression-engine/) 4 | 5 | ## System Architecture Progress 6 | 7 | Building on July's architectural foundation, August focused on implementing the core comparison engine and developing platform-specific adapters. The project has undergone some changes after discussions regarding the inclusion of external dependencies, aiming to avoid them where possible. After further discussion, a significant change was made to the project's approach. For this, the executable binaries option which was introduced earlier was explored in more detail. Finally, after extensive research, instead of creating a modular component used by both p5.js and Processing, a custom comparison algorithm was implemented directly in Kotlin for the Processing environment. 8 | 9 | The core architectural components established now include a custom pixelmatch-based comparison engine with threshold management systems implemented in Kotlin for Processing and the reporting system involved. 10 | 11 | ## Work Accomplished 12 | 13 | ### Comparison Engine Implementation 14 | Successfully completed the core comparison engine implementation. This would have been the foundational component that both Processing and p5.js would have utilized through their respective adapters, with a new focus on platform-specific implementations. You can have a look at that [here](https://www.npmjs.com/package/visual-regression-engine) 15 | 16 | **Technical Implementation Details:** 17 | * **Pixelmatch integration** - Implemented the proven pixelmatch algorithm, adapting it to work within a package structure that can be consumed. The integration maintains the same threshold logic and decision-making processes that have been tested and refined in the p5.js ecosystem. 18 | * **Threshold management system** - Developed a configurable threshold system that allows developers to set acceptable levels of visual variation based on their specific testing needs. This system handles false positives by providing granular control over pixel difference tolerance, color variance thresholds, and anti-aliasing detection. 19 | * **Cross-platform image handling** - Implemented robust image processing capabilities that work consistently across different operating systems and image formats, ensuring that visual comparisons produce identical results regardless of the development environment. 20 | 21 | ### Binary Development Investigation and Technical Challenges 22 | Conducted extensive research into binary development approaches for creating standalone executables that could eliminate Node.js runtime dependencies. This investigation was motivated by the desire to reduce contributor setup friction and create more portable testing tools. Have a look at the releases [here](https://github.com/Vaivaswat2244/visual-regression-engine/actions) 23 | 24 | **Nexe Implementation Challenges:** 25 | * **Massive build overhead** - Nexe compilation required building the entire Node.js runtime, resulting in build times of approximately 40 minutes per compilation. This proved impractical for iterative development and CI/CD workflows. 26 | * **Parameter deprecation issues** - The build process failed due to deprecated parameters in the fetch mechanism used to download Node.js sources. The deprecated API endpoints caused silent failures that were difficult to diagnose, often resulting in corrupted or incomplete builds. 27 | * **Binary size concerns** - Even successful Nexe builds produced executables exceeding 100MB due to the embedded Node.js runtime, making distribution cumbersome for library developers who expect lightweight testing tools. 28 | 29 | **Esbuild Compilation Limitations:** 30 | * **Runtime dependency persistence** - Despite esbuild's excellent bundling capabilities, the output JavaScript files still required Node.js runtime for execution, completely defeating the purpose of creating standalone binaries for reduced dependency management. 31 | * **Module resolution complexities** - Esbuild struggled with dynamic imports and certain Node.js built-in modules used by pixelmatch and image processing libraries, requiring extensive configuration workarounds that introduced additional maintenance overhead. 32 | 33 | **pkg Package Challenges:** 34 | * **Deprecation status complications** - Although pkg is officially deprecated, investigation continued to ensure comprehensive evaluation of available options. The deprecated status raised concerns about long-term maintainability and security updates. 35 | * **Windows-specific build failures** - Encountered consistent compilation errors on Windows platforms, specifically related to native module compilation and path resolution. These errors proved particularly challenging to debug due to pkg's limited error reporting and the deprecated nature of the toolchain. 36 | * **Native dependency handling** - pkg struggled to properly bundle native dependencies used by image processing libraries, often resulting in runtime errors when the binary attempted to access missing native modules. 37 | 38 | **Deno Runtime Evaluation:** 39 | * **Dependency ecosystem limitations** - Deno's approach to dependency management conflicted with the project's need to leverage existing NPM packages like pixelmatch. Converting to Deno-compatible imports would require significant refactoring of established, well-tested libraries. 40 | * **Node.js compatibility layer issues** - When attempting to use Node.js compatibility mode for NPM dependencies, Deno essentially required Node.js compilation anyway, eliminating the benefits of a standalone runtime and creating additional complexity without solving the original dependency problem. 41 | * **Library integration challenges** - The fundamental mismatch between Deno's security model and the project's need to integrate with existing Processing and p5.js toolchains created architectural conflicts that would require extensive workarounds. 42 | 43 | ### NPM to Kotlin Implementation of Comparison Algorithm 44 | 45 | The changing of the project's revised scope involved replacing the `pixelmatch` NPM package with a custom Kotlin function that calculates the exact Euclidean distance for color differences using the formula `sqrt((dr² + dg² + db² + da²)) / 255.0`. This ensures a highly accurate and consistent comparison. The rest of the implementation, including threshold management, image scaling, and diff image generation, was then structured similarly to the original NPM-based design, but entirely within the Kotlin ecosystem. This allows for a completely self-contained solution for Processing, eliminating the need for Node.js runtime or NPM dependencies. 46 | 47 | ## Challenges Encountered 48 | 49 | The primary technical challenge was the binary development investigation, which consumed significant development time while ultimately proving unviable for this project's requirements. Each binary compilation approach presented unique technical hurdles that made them unsuitable for the contributor-friendly, cross-platform solution needed for Processing visual testing. 50 | 51 | Cross-platform testing revealed subtle inconsistencies in image rendering and file system behavior that required platform-specific handling within the adapters, particularly around font rendering differences and color space management between operating systems. Additionally, considerable thought was put into ensuring the Kotlin implementation of the pixelmatch algorithm would be as robust and accurate as its JavaScript counterpart. 52 | 53 | ## Key Insights 54 | 55 | **Binary development trade-offs proved prohibitive** - The investigation into standalone binary creation revealed that current JavaScript-to-binary compilation tools introduce more complexity than they solve. The compilation overhead, platform-specific issues, and maintenance concerns outweigh the potential benefits of eliminating Node.js as a runtime dependency. 56 | 57 | **Kotlin implementation for Processing allows for dependency-free solution** - Implementing the core comparison logic directly in Kotlin for Processing provided a clean, dependency-free solution that integrates natively with the Processing environment, avoiding the complexities of bridging between NPM and Gradle ecosystems. 58 | 59 | **Image comparison threshold tuning is critical** - Automated visual regression testing requires careful threshold configuration to balance sensitivity with practicality. Different types of Processing sketches may require different tolerance levels for meaningful regression detection. 60 | 61 | ## September Plans 62 | 63 | September will focus on completing the adapter implementations and beginning comprehensive testing with actual Processing development workflows. 64 | **Processing Implementation and Reporting:** 65 | * Develop comprehensive test suites for the Kotlin implementation to ensure reliability across different image types and scenarios. 66 | * Create Processing-specific testing framework integration that leverages the native Kotlin comparison engine. 67 | * Design and implement a reporting system for Processing that provides clear visual regression test results and integrates with Processing's development workflow. 68 | 69 | **Integration Testing and Validation:** 70 | * Conduct extensive testing with real-world Processing libraries to validate the adapter functionality and identify integration issues. 71 | * Performance optimization and memory usage analysis for both adapters under typical development workflows. 72 | 73 | The primary deliverable for September will be fully functional adapters that Processing developers can integrate into their existing testing workflows with minimal setup friction, along with comprehensive documentation and migration guides. -------------------------------------------------------------------------------- /2025_BuildingBridges/monthly-reports/pr05_2025_July_report_Vaivaswat_Dubey.md: -------------------------------------------------------------------------------- 1 | # July Monthly Report: Snapshot Testing for Processing and p5.js 2 | By **Vaivaswat Dubey** 3 | 4 | - [Project Repository: Processing Comparison Engine](https://github.com/Vaivaswat2244/processing-comparison-engine/) 5 | 6 | ## System Architecture and Shared Components 7 | 8 | ### How the System Works 9 | 10 | The proposed architecture centers around a shared comparison engine that both Processing and p5.js can utilize through platform-specific adapters. Rather than forcing either ecosystem to change their existing workflows, each platform maintains its current testing approach while leveraging a common underlying image comparison service. 11 | 12 | ### Shared Components from p5.js 13 | 14 | The core comparison functionality will be extracted from p5.js's existing visual testing infrastructure and packaged as a reusable NPM module: 15 | 16 | * **Proven pixelmatch implementation** - The existing pixelmatch algorithm setup that p5.js currently uses for visual regression testing, including all the threshold logic and decision-making around what constitutes a meaningful visual difference. 17 | 18 | * **Threshold management system** - The established approach for handling false positives and determining acceptable levels of visual variation, which has already been tested and refined in the p5.js ecosystem. 19 | 20 | ### Platform-Specific Adapters 21 | 22 | Instead of creating a monolithic system, thin adapter layers will handle ecosystem-specific integration: 23 | 24 | * **Java adapter for Processing** - Utilizes gradle-npm plugin to interface Processing's Gradle-based build system with the NPM-packaged comparison engine, allowing Processing libraries to perform visual testing without changing their existing build workflows. 25 | 26 | * **JavaScript adapter for p5.js** - Works within the existing Vitest testing framework, serving as a bridge between p5.js's current test structure and the shared comparison engine. 27 | 28 | This adapter approach ensures that improvements and bug fixes to the core comparison logic benefit both ecosystems, while allowing each platform to evolve their testing approaches independently without breaking the other. 29 | 30 | ## Work Accomplished 31 | 32 | The main goal this month was to explore and implement proof-of-concept integration solutions between NPM and Gradle build systems, while researching containerization alternatives and binary development approaches. These specific approaches were selected based on key design criteria: minimizing contributor friction, maintaining compatibility with existing Processing and p5.js workflows, and enabling shared development efforts between the two ecosystems. Initial work focused on understanding existing integration patterns in the community and evaluating different technological approaches for my specific use case. 33 | 34 | ### NPM-Gradle Integration Proof of Concept 35 | 36 | I successfully developed a working proof of concept for NPM-Gradle integration that demonstrates seamless coordination between JavaScript/Node.js and JVM-based build processes. This proof of concept addresses the common challenge of managing hybrid application dependencies across different package managers. NPM was ultimately chosen as the preferred approach due to its significantly lower contributor setup friction compared to containerized alternatives. 37 | 38 | The integration developed includes: 39 | 40 | * **Custom Gradle tasks integration** - Created automated Gradle tasks that invoke NPM commands during the build lifecycle, ensuring frontend dependencies are resolved and assets are compiled as part of the standard Gradle build process. This eliminates the need for developers to manually run separate NPM commands and reduces the chance of build inconsistencies. 41 | 42 | * **Unified dependency management** - Implemented coordination between package.json and build.gradle files so that both JavaScript and JVM dependencies are managed through a single build command. This includes automatic detection of dependency conflicts and version alignment between the two package management systems. 43 | 44 | * **Cross-platform build compatibility** - Developed build scripts that work consistently across Windows, macOS, and Linux development environments without requiring platform-specific configuration. This ensures that all team members can contribute regardless of their operating system choice. 45 | 46 | ### Gradle Setup Research 47 | 48 | Conducted extensive research on optimal Gradle configurations for modern development workflows, focusing on build performance optimizations and plugin ecosystem compatibility. Key findings included the importance of proper daemon configuration, build caching strategies, and plugin selection for hybrid JavaScript/JVM projects. 49 | 50 | ### Docker Alternatives Investigation 51 | 52 | Researched containerization technologies including Docker. However, after evaluation, the Docker approach was ultimately declined due to the significant contributor friction it would introduce. Requiring contributors to learn container technologies and manage containerized development environments would create unnecessary barriers to community participation and make the contribution process more complex than necessary. 53 | 54 | ### Binary Development with Go and Rust 55 | 56 | Comprehensive analysis was performed comparing Go and Rust for binary development, evaluating compilation speed, runtime performance, memory management, and development velocity. However, questions arose regarding how well these binary approaches would integrate with the existing Processing environment and whether they would create additional complexity for library developers who are already familiar with Java-based workflows. 57 | 58 | ### Comparison Engine Development 59 | 60 | I began development of a comparison engine using pixelmatch, a JavaScript library for precise pixel-level image comparison. This work includes: 61 | 62 | * Established the foundation for automated visual testing by integrating pixelmatch into my testing infrastructure. This JavaScript library provides precise pixel-level comparison capabilities that can detect even subtle visual differences in Processing library outputs, making it ideal for regression testing and quality assurance workflows where images are generated directly in p5.js and Processing environments. 63 | 64 | * Implemented the algorithm which is already in place in p5 with some changes to work with the npm structure. 65 | 66 | ## Challenges Encountered 67 | 68 | The main challenge met in this work was navigating the complexity of Gradle's plugin system and build lifecycle. The documentation was often unclear about what specific plugins accomplish, particularly when trying to implement solutions using Kotlin DSL instead of the traditional Groovy syntax. I encountered unexpected difficulties where builds would succeed in IntelliJ's Gradle integration but fail when executed from the command line, requiring investigation into classpath and environment differences. 69 | 70 | Another significant challenge was balancing technical capabilities with contributor accessibility. While containerized solutions offered better isolation and reproducibility, they would require contributors to learn additional tools and concepts. Similarly, while Go and Rust offered performance benefits for binary development, their integration with the Processing ecosystem remained unclear and could potentially fragment the development experience. 71 | 72 | ## Key Insights 73 | 74 | Through my research and development work, several important insights emerged: 75 | 76 | * **Contributor friction is a critical factor** - Technical solutions must be evaluated not just on their capabilities but on how they impact the ease of contribution. NPM integration was preferred over Docker specifically because it requires less setup and learning from contributors. 77 | 78 | * **NPM-Gradle integration provides the best balance** - This approach offers improved build automation and dependency management while maintaining familiar workflows for developers already working with JavaScript and Java technologies. 79 | 80 | * **Processing environment integration concerns** - Binary development approaches using Go and Rust, while technically promising, raise questions about how well they would integrate with Processing's existing Java-based ecosystem and whether they would create unnecessary complexity for library developers. 81 | 82 | * **Build system complexity requires careful documentation** - The fragmented nature of Gradle documentation, particularly around Kotlin DSL usage, creates barriers that must be addressed through comprehensive setup guides and examples. 83 | 84 | ## August Plans 85 | 86 | The plans for next month are focused on two main areas: 87 | 88 | 1. **Comparison Engine and Testing Infrastructure** - I will continue development of the pixelmatch-based comparison engine which will involve testing it with the images generated in the p5 and processing environment. This includes establishing tolerance thresholds and acceptance criteria for visual differences, determining what percentage of pixel differences should trigger test failures versus passes, and creating a standardized approach for handling false positives in automated visual testing. 89 | 90 | 2. **Adapter Development** - I plan to complete the initial implementation of both the processing and p5 adapter which involve building the visual testing infrastructure for both the project. Given that p5.js already uses Vitest and has integrated visual tests, while Processing will require a complete setup from scratch, I will prioritize the Processing adapter development to establish the foundational infrastructure needed for that ecosystem. 91 | 92 | In August I will be finishing the comparison engine implementation and will start testing these automated comparison systems with actual Processing and p5.js development workflows to ensure they can effectively catch regressions and visual inconsistencies. -------------------------------------------------------------------------------- /2025_BuildingBridges/monthly-reports/pr05_2025_August_report_Claire_Peng.md: -------------------------------------------------------------------------------- 1 | # Monthly Report - August: Incremental Typescript Migration for the p5.js Web Editor 2 | 3 | ## Overview 4 | 5 | Over August, the PRs from July were reviewed, and received really useful feedback to action. As such I spent a great portion of the month going back to old PRs, updating to address feedback, and updating new PRs to use the learnings from these previous PRs. The learnings will be detailed below. 6 | 7 | In addition to the refining the PRs from July, I have additionally opened PRs to: 8 | 9 | - [Update the README to introduce the migration](https://github.com/processing/p5.js-web-editor/pull/3581) 10 | - [Migrate the entire client/components folder](https://github.com/processing/p5.js-web-editor/pull/3619) 11 | - [Migrate the root files related to redux, and migrate an instance of a redux reducer system (preferences system) -- draft PR](https://github.com/processing/p5.js-web-editor/pull/3633) 12 | 13 | As of the end of August, the following have officially been merged into the codebase: 14 | 15 | 16 | 17 | ## Key Feedback & Typescript Styling Decisions: 18 | 19 | ### Use `interface` instead of `type` for defining props: 20 | 21 | ```tsx 22 | // ✅ interface 23 | interface ButtonProps { 24 | label: string 25 | disabled?: boolean 26 | onClick: () => void 27 | } 28 | const Button = ({ label, disabled, onClick }: ButtonProps) => (...) 29 | 30 | // ❌ type 31 | type ButtonProps = { 32 | label: string 33 | disabled?: boolean 34 | onClick: () => void 35 | } 36 | 37 | // ✅ keep type for primitives 38 | type UserId = string; 39 | ``` 40 | 41 |
42 | Why? 43 | 44 | - Clearer semantic meaning when describing structured objects vs. primitives 45 | 46 |
47 | 48 | ### Prefer named exports over default exports 49 | 50 | ```tsx 51 | // ✅ named 52 | export const Button = () => (...) 53 | import { Button } from "./Button" 54 | 55 | // ❌ default 56 | export default Button 57 | import Button from "./Button" 58 | ``` 59 | 60 |
61 | Why? 62 | 63 | - Named exports make it easy to see what's exported and imported at a glance. 64 | 65 | - Avoid renaming confusion and mismatched imports (e.g., import X from ... when the original name differs). 66 |
67 | 68 | ### Use `enum` instead of union types for prop variants 69 | 70 | ```tsx 71 | // ✅ Enum for button variant 72 | enum ButtonVariant { 73 | Primary = "primary", 74 | Secondary = "secondary", 75 | } 76 | interface ButtonProps { 77 | variant: ButtonVariant; 78 | } 79 | 80 | // ❌ Union type for button variant, not as preferrable. 81 | type ButtonProps = { 82 | variant: "primary" | "secondary"; 83 | }; 84 | ``` 85 | 86 |
87 | Why? 88 | 89 | - Enums offer explicit, centralized definitions of allowed values. 90 | 91 | - They prevent typo-related bugs and can be extended easily, often aligning better with team-wide or design-token systems. 92 |
93 | 94 | ## Challenges, Progress & Key Decisions: 95 | 96 | This month was quite challenging, as I am not as experience with React and React Typescript syntax. I struggled a lot with figuring out how to resolve type-errors with `React.forwardRef` especially, which came up a lot with `useModalClose` and the components that used it. I am definitely still in the progress of learning what `refs` and `forwardRefs` are in React. 97 | 98 | Through this process, I also learned more about `React.HTMLAttributes` and how React defines **base props** for native HTML elements. Extending from these base props allows custom components to inherit all standard attributes (like `id`, `className`, `style`, etc.) while layering on their own specific props. This makes it easier to build flexible, reusable components without losing type safety for the important props. Having only done React with JavaScript in the past, it was interesting to learn about the source of these attributes and see them explicitly declared in TypeScript, which made it much clearer how props are inherited and extended in custom components. 99 | 100 | Another thing that came up was learning how to infer the base React component's props to attach to a component and how to overwrite its props with `Omit`. A concrete example of this was the `/client/common/ButtonOrLink` component: this component renders either a ` 195 | ); 196 | } 197 | ); 198 | ``` 199 | 200 | I made the same decision for the `/client/common/Button` component, extending `React.HTMLAttributes` while layering on explicit props like `href`, `to`, `kind`, `display`, and `iconOnly`. This approach allowed the components to remain flexible for legacy usage while still providing type safety for the most important props. An improvement in the future, once more of the code is migrated, would be to update these components with stricter types to enforce valid prop combinations and give stronger guarantees to consumers. 201 | 202 | One key decision that came up upon discussion with my mentor Connie, and p5.js Web Editor project lead, Rachel, was that we would de-prioritise unit testing for time. In the previous month, I was attempting to add unit tests to each component touched, but this was quite time intensive, and we decided going forward that we would only add unit tests for more complex components. I may revisit this in October after more of the codebase is migrated and I feel more comfortable with React TS. 203 | 204 | Throughout the process this month, I carefully checked instances of useage for each component or function up for migration, and was able to delete a number of unused modules. This was super satisfying and helped clean up the codebase a lot. 205 | 206 | ## PR's: 207 | 208 | ### Updated from previous month to address feedback & now merged: 209 | 210 | - [pr05 Typescript Migration 3: Migrate the client/utils folder](https://github.com/processing/p5.js-web-editor/pull/3553) 211 | - [pr05 Typescript Migration #5: Migrate client/common folder](https://github.com/processing/p5.js-web-editor/pull/3565) 212 | 213 | ### Current Month PR's - Merged: 214 | 215 | - [pr05 Typescript Migration #6: Initial docs on typescript migration](https://github.com/processing/p5.js-web-editor/pull/3581) 216 | - [pr05 Typescript Migration #7: migrate client components](https://github.com/processing/p5.js-web-editor/pull/3619) 217 | 218 | ### Current Month PR's - In Draft or In Review: 219 | 220 | - [pr05 Typescript #8: migrate client/components/Menubar/MenubarSubmenu](https://github.com/processing/p5.js-web-editor/pull/3623) 221 | - I found this component really challenging so I had to do it on a separate PR from the rest of the client/components folder. 222 | - [pr05 Typescript #9: Migrate redux root files & migrate instance of redux reducers/actions (preferences)](https://github.com/processing/p5.js-web-editor/pull/3633) 223 | 224 | ## Next Steps: 225 | 226 | Going forward, I now know the time-in addressing feedback on open PR's -- I shouldn't consider these 'done' till all feedback is addressed and merged, and I should expect some time to do this as I'm still learning best practices for TS. 227 | 228 | ### September goals: 229 | 230 | - Address any feedback for outstanding in-review & in-draft PRs 231 | - Set up TS dependencies for the server folder 232 | - Migrate as much as possible from the following for the server folder: 233 | - server root folder 234 | - server/views folder 235 | - server/models folder 236 | - server/routes folder 237 | - server/controllers folder (nice-to-have, will take the most time & probably will spill into October) 238 | -------------------------------------------------------------------------------- /2024_NewBeginnings/final-reports/pr05_2024_final_report_Claudine_Chen.md: -------------------------------------------------------------------------------- 1 | # Final Report for Simplifying the workflow for Processing libraries 2 | 3 | | **Project** | [Simplifying the workflow for Processing libraries, tools, and modes](https://github.com/processing/pr05-grant/wiki/2024-Project-List-for-%60pr05%60-=-Processing-Foundation-Software-Development-Grant#%F0%9F%85%BF%EF%B8%8F-simplify-the-workflow-for-processing-libraries-tools-and-modes) | 4 | | :--- | :--- | 5 | | **Grantee** | [Claudine Chen](https://github.com/mingness) | 6 | | **Mentor** | [Stef Tervelde](https://github.com/Stefterv) | 7 | | **Repos**| https://github.com/processing/processing-library-template
https://github.com/processing/processing-contributions
documentation repo: https://github.com/mingness/pr05-simplify-workflows| 8 | 9 | 10 | ## Technical Decisions 11 | 12 | ### Library template 13 | 1. The Gradle build file written in Kotlin, for the library template. Kotlin is now the default script language. This decision, however, forced us to put everything in one file, rather than having editable fields in one file, and release tasks in another. This is because Kotlin doesn't allow the build file to be split into multiple files. 14 | 2. Gradle is not easy, which is probably why the template `processing-library-template-gradle` seemed to take a 15 | design approach of eliminating any interactions by the user with Gradle scripts, opting instead to have the user input all relevant build parameters into a config file. Propagating the information from this config file into Gradle tasks required 16 | additional code. So while this approach does work, it comes at a cost of readability of the Gradle scripts. We opted to not protect the user from Gradle, but let this be an opportunity to learn how to interact with Gradle. We tried to simplify the interaction as much as possible, and to instruct and explain as much as possible. In our template, the user will add their library name, domain, and dependencies into the Gradle file, guided by comments. 17 | 3. We had a principle, to define values only once. Therefore, the version is defined in Gradle once, and propagated to the library properties file. Previously the version was also reported back by the library, but all the options available to accomplish this without setting the version twice were unappealing, and we chose to not implement this. 18 | 4. We chose to promote the use of MkDocs for creating documentation websites via Github project pages. This provides straightforward documentation websites based on markdown. Hosting documentation is a requirement for publishing a library in the Contribution Manager. The old documentation recommended people build and host their own documentation website which adds a lot of complexity. The main goal of this project was to simplify the process of creating and publishing a library, so it made sense to automate the process of creating/hosting this documentation as much as possible. 19 | 5. We chose to have a `release.properties` file, where the user will input values for the library properties text file. All fields are mapped directly, and the `version` in the 20 | Gradle build file is mapped to `prettyVersion` in the properties file. 21 | 22 | ### Add contributions workflow 23 | 1. We decided to make a new database file, called `contributions.yaml`. The format of the database file was selected to be a list of objects, in yaml format. Yaml format allows for ease of understanding any edits in the database. Each edit will appear per line with the field name, making each edit easy to see using Github's interface, compared to tabular formats, like csv. For tabular formats, Github's interface, which shows if a row has an edit, would highlight more which contribution had and edit, but the edit itself would be harder to see, especially since the column label would be on a different line. 24 | 2. In the previous repository, some data from the properties files are overwritten by design, based on the categories the source url was listed under, or if it's a "broken" contribution. To make it intuitive to compare values in the database against the properties file, it was decided that the data in the database directly reflect the data in the properties files. To be able to overwrite this data in the output files, we've implemented an `override` field. The value of the `override` field is an object, where any component field values will replace the existing field values. For example, libraries in the `broken.conf` file are outdated, and we want to cap the `maxRevision` to `228`. This cap can be applied by setting `override` to {`maxRevision`: `228`} 25 | 3. We want the database file to contain all contributions ever. This means, we need to store contributions that are deprecated, and we need to store that state. This is set by a new `status` field. If we know a library has been deprecated, then we can manually set the `status` to `DEPRECATED` and updates for library will no longer be attempted. A value of `VALID` indicates the library is live. `BROKEN` indicates that the url did not serve the properties file as expected, but we will continue to check the source url for the properties. This `status` value is automatically set if the properties are not found at the url during an update. 26 | 4. The previous process required the new contributor to email the Processing librarian. 27 | We have replaced this initial contact with an issue form. The issue form makes it 28 | clear as to what is required to register the library, and can be used to trigger workflows. 29 | 5. The creation of the issue form triggers a Github workflow, which automatically 30 | retrieves the properties file, parses and validates it, and if valid, creates a 31 | pull request for the Processing librarian to review. The way the workflow was designed, 32 | was to allow for a rerun of the workflow, in case of edits to the repository. In order to do this, the branch is deleted before being recreated again by the workflow. 33 | 6. To ensure readability and to use preexisting documentation, use of third-party 34 | actions was preferred. 35 | 7. The workflow that checks the urls daily needs to merge its changes automatically, without human intervention. For this reason, we make direct changes into the default branch `main`. This workflow will check the url of every contribution that doesn't have a `status` of `DEPRECATED`, and will update the contribution if the `version` changes. If there isn't a properties file at the url, the contribution will be marked with a status of `BROKEN` until it is live again. 36 | 8. In the new repository, the output files are no longer a part of the repository, 37 | to make source of truth clear - the database file. These output files will instead 38 | reside as workflow artifacts. 39 | 9. Two different levels of validation are used, where the validation for new contributions is more strict than for existing contributions. This is because some older properties files have different field names than currently expected. Currently we look for `authors` and `categories`, but in older files they might be `authorList` or `category`. 40 | 10. We implemented validation using Pydantic's built in validation, simply because considering all the cases manually became too much. Pydantic wasn't used from the start, because it was intended to write the scripts in a language in the Java family, like Kotlin. However, they currently are in Python. 41 | 42 | 43 | ## Challenges 44 | 45 | - Writing the Gradle build file was a challenge. Gradle is not well documented, and while I had used it before, coding in it is another matter. Also, we chose to write the build file in Kotlin, for which there is even less documentation. 46 | - Gradle does not work well when run from command line. I had an apparent issue with caching when running Gradle from command line. It worked perfectly from the Gradle menus in Intellij and VS Code. 47 | - There were some difficulties in implementing the contributions database file, simply because, data is messy, and is rarely as theoretically clean as expected. This work needed to convert logic enshrined in config files and a script into data. The process of doing this required iterations of trial and error. 48 | 49 | 50 | ## Tasks Completed 51 | 52 | ### Library template 53 | 54 | - [x] A Gradle build file, that 55 | - uses jitpack to resolve Processing, instead of local jar files. Once Processing is offered on Maven, this will be changed to resolve via official sources. 56 | - has Gradle tasks for building the library, creating the properties file, creating all release artifacts, and installing the library in a local Processing instance. 57 | - [x] The template contains an example library that uses an external dependency, as an a functioning example to work from. The template can be compiled as is. This template provides an example that can be easily accessed in Processing from the examples menu. 58 | - [x] The template has a workflow that includes the necessary release artifacts in a Github release. The default release artifacts only include the source code. 59 | - [x] It provides a new mkdocs powered documentation website example. This framework provides a simple format suitable for documentation websites, based on markdown files. 60 | - [x] The template includes documentation, including detailed instructions on getting started, using the Gradle build file, and how to release and publish. 61 | - [x] The library template is now integrated into the processing account. 62 | - The previously named `processing/processing-library-template` is now named 63 | `processing/processing-library-template-ant`. 64 | - The previously named `mingness/processing-library-template` is now named 65 | `processing/processing-library-template`. 66 | 67 | ### Add contributions workflow 68 | - [x] Create new database file, called `contributions.yaml`, as designed. The design is described in https://github.com/mingness/pr05-simplify-workflows/blob/main/Adding_contribution_workflow_notes.md#design-of-database-file 69 | - [x] Write script for checking online property txt files for each contribution. 70 | - [x] Create scripts to create `pde/contribs.txt`, and `content/contributions/*.json` files. 71 | - [x] Create issue form for new library contributors to fill, which then automatically creates the pull request that changes the database file. 72 | - [x] Test output files with website and contribution manager. 73 | - [x] Create workflow such that, if there is an update, edit the database file, and recreate contribs.txt and json files. The output files are stored as workflow artifacts. 74 | - [x] The processing contributions repository is now integrated into the processing account. 75 | - The previously named and private repository `processing/processing-contributions` is now named `processing/processing-contributions-legacy`. 76 | - There is now a repository named `processing/processing-contributions`, which contains 77 | only the important files of `mingness/processing-contributions-new`. 78 | 79 | 80 | ## Limitations 81 | 82 | - The library template at `processing/processing-library-template` has been tested only for a simple case. We will have to wait for a crowd of people to test the template, and hopefully let us know if something is not working well. 83 | - The scripts in `processing/processing-contributions` do not have tests. 84 | - The `contributions.yaml` database file in `processing/processing-contributions` has three fields which do not have comprehensive data. These fields are `dateAdded`, `dateUpdated`, and `previousVersions`. To fill in this data will require delving into the archives the Github commit history. 85 | - Only a template for libraries was created. A template for tools and modes was not investigated. 86 | - Originally there was an idea to run the examples from the library template using the IDE, but this was not implemented. However, it is possible to run the examples in Processing, so perhaps this is not a limitation. 87 | 88 | ## Work Remaining & Next Steps 89 | 90 | - [ ] Write tutorial, and publish on Processing website 91 | - [ ] Compare the output files from the daily update workflow of `processing/processing-contributions` with the output files from `processing/processing-contributions-legacy` 92 | - [ ] Use the output files from the daily update workflow of `processing/processing-contributions` for Contribution Manager and Processing website 93 | - [ ] Review feedback on the library template at `processing/processing-library-template`, and iterate on improvements 94 | - [ ] Delve into archives of contributions, and add data for fields `dateAdded`, `dateUpdated`, and `previousVersions` in the `contributions.yaml` database file in `processing/processing-contributions`. 95 | -------------------------------------------------------------------------------- /2024_NewBeginnings/final-reports/pr05_2024_final_report_Diya_Solanki.md: -------------------------------------------------------------------------------- 1 |

2 | pr05'24 Final Report 3 |

4 |

Create an Official Processing VSCode Extension 5 |

6 | 7 | | **Project** | [Processing Language Server Extension](https://github.com/processing/pr05-grant/wiki/2024-Project-List-for-%60pr05%60-=-Processing-Foundation-Software-Development-Grant#%F0%9F%85%BF%EF%B8%8F-create-an-official-processing-vscode-extension) | 8 | |-----------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| 9 | | **Grantee** | [Diya Solanki](https://github.com/diyaayay) | 10 | | **Mentors** | [Sam Lavigne](https://lav.io/), [Justin Gitlin](https://cacheflowe.com/) | 11 | | **Repository** | [Processing-language-server-extension](https://github.com/diyaayay/processing-language-server-extension) | 12 | 13 | 14 | ## Description 15 | 16 | This report details my work, development processes, and insights gained during the pr05 2024 "New Beginnings" software development grant with the [Processing Foundation](https://processingfoundation.org/). During the past four months, I have focussed on developing the [Processing Language Server Extension](https://github.com/diyaayay/processing-language-server-extension/) for the Processing software for VSCode. Currently, Processing is distributed as a standalone software compatible with multiple platforms; this extension integrates `.pde` sketch files into VSCode, providing robust IntelliSense support to enhance the coding experience. 17 | It was a privilege to work under the mentorship of Sam Lavigne and advisor Justin Gitlin. Together, we conducted bi-weekly project check-ins to review progress and address challenges in development. 18 | The Language Server Extension includes a server-side implementation and a client interface within the VSCode editor, delivering IntelliSense functionality to developers working with .pde sketches. The complete project plan is accessible within the project's [roadmap](https://github.com/diyaayay/processing-language-server-extension/blob/Development/ROADMAP.md). 19 | 20 | 21 | ## What code got merged in 22 | - [Run Processing Sketches](https://github.com/diyaayay/processing-language-server-extension/blob/Development/client/src/sketchRunner.ts):  23 | The Run and Stop button toggle according to the state of the sketch. The sketchbook can contain multiple sketches each residing in a directory of the same name as that of the sketch. 24 | - [Grammar](https://github.com/diyaayay/processing-language-server-extension/tree/Development/server/src/grammar/terms): Defines Processing specific grammar model to parse the textDocument in workspace. 25 | - [Parser](https://github.com/diyaayay/processing-language-server-extension/blob/Development/server/src/parser.ts): The parser.ts file defines functions to parse Java code into an Abstract Syntax Tree (AST) and extract tokenized information from the code. The main function, parseAST, processes Java code and returns an array of token pairs representing each parse tree node and its parent. The lineMap function creates mappings of each word in a line to its start and end positions, helping locate tokens within specific lines. The parse function leverages the java-ast library to handle parsing while muting unnecessary console output through a redirectConsole utility, which temporarily disables logging during the parsing process. 26 | - [Sketch Runner](https://github.com/diyaayay/processing-language-server-extension/blob/Development/client/src/sketchRunner.ts): Defines a SketchRunner class that manages the execution of Processing sketches in a Visual Studio Code extension. It provides functionality to initialize the sketch runner with a specified Java Runtime Environment (JRE) path and a sketch directory. The class can run and stop sketches while handling output from the Processing environment through buffered channels to manage the display in the VSCode output window. It also includes error logging to capture issues during execution and during the copying of compiled sketches. The copyCompiledSketch method ensures that .class files are correctly transferred from the source to the destination directory, maintaining the structure of the original sketch. Additionally, the dispose method cleans up resources when the sketch runner is no longer needed, preventing memory leaks. Overall, the class encapsulates the functionality required to run and manage Processing sketches effectively within the editor environment. 27 | - [Sketch Errors](https://github.com/diyaayay/processing-language-server-extension/blob/Development/client/src/sketchRunner.ts): On Running a sketch, the LSP extension starts watching the sketch and logs errors in the Output tab. 28 | - [On-Hover Documentation](https://github.com/diyaayay/processing-language-server-extension/blob/Development/server/src/hover.ts): The on-hover feature would display the documentation for every keyword of the Processing language stored in [XML files](https://github.com/diyaayay/processing-language-server-extension/tree/Development/server/out/processing/lspinsights). 29 | - [Code Completion](https://github.com/diyaayay/processing-language-server-extension/blob/Development/server/src/completion.ts): Keywords are reserved keywords in the Processing language. This feature would display a pop-up menu containing a list of keywords aligning with the triggering character. 30 | - [Goto Reference](https://github.com/diyaayay/processing-language-server-extension/blob/Development/server/src/references.ts): This feature responds to the onReferences requests by the user when they select the option of, found in the context menu to navigate to the schemas that reference the location under the cursor. 31 | - [Goto Definition](https://github.com/diyaayay/processing-language-server-extension/blob/Development/server/src/definition.ts): This is vice-versa of the former issue i.e., jump to the schema a reference refers to. This is the same as going to the variable declaration through `Ctrl/Cmd + Click` or the context menu. This uses the `onDefinition` request from the user in the extension host to redirect the cursor to the definition the reference refers to. 32 | 33 | ## Techincal challenges and decisions taken: 34 | 35 | - **Running Processing headless** without requiring the user to set paths or require some complex setup was the main challenge to make the version migration work everywhere. 36 | - This required quite a bit of research and decisions taken regarding bundling everything in the extension:- We aimed to run the processing sketches inside the extension host (which is different from running the sketches in a JavaScript frontend, for example). We could simply call the binary using some `npm` package like `child_process`, `spawn-process`, etc., but then we need it inside the extension host as it should also have the client-server support - necessary to make all LSP features work smoothly. 37 | - For example, [vscode-languageserver-node](https://github.com/microsoft/vscode-languageserver-node) is using a custom script called `symlink` which can be executed normally as `npm run symlink`. It just executes a file called [symlink.js](https://github.com/microsoft/vscode-languageserver-node/blob/main/build/bin/symlink.js) which creates the hardlinks and softlinks between the protocol, the server, the client and the extension folder. Some languages have better low-level control, like [C](https://pubs.opengroup.org/onlinepubs/009696899/functions/symlink.html#:~:text=The%20symlink()%20function%20shall,contained%20in%20the%20symbolic%20link) and [Rust](https://doc.rust-lang.org/stable/std/os/unix/fs/fn.symlink.html) that support links creation. 38 | - Opting for a similar approach, the Processing distribution is binded to the extension. The large size of the extension is due to custom dependencies downloaded via the [dependency.bat](https://github.com/diyaayay/processing-language-server-extension/blob/Development/dependency.bat) script. The [dependency.zip](https://github.com/diyaayay/processing-language-server-extension/releases/download/v1.2/dependency.zip) contains the Processing distribution for Windows, which is extracted into `jre/deps`. 39 | - **Release binding with commit**: I encountered an unexpected issue with GitHub releases that took me some time to understand. When the maintainer creates a GitHub release, it automatically binds to a commit on the default branch. When this branch is cloned, it causes multiple issues, including connection failures with the LSP client and errors related to the VSCode API. After some research, I resolved the problem by binding GitHub releases to a commit on a branch other than the default branch. 40 | 41 | ## Current state of the Project: 42 | 43 | As outlined in the project roadmap, the LSP extension has been developed for cross-platform compatibility. Packaging Processing's latest version to bundle with the extension and run from its server was achieved by creating symlinks using Rust (a low-level language) for **Windows**. This eliminated the need for users to add the Processing path to their environment variable and enabled Processing to run in headless mode. This method in the future would enable the distribution to run directly from the extension on all platforms. The following milestones have been successfully achieved: 44 | 45 | - [x] Stable Architecture 46 | - [x] .vsix release and VSCode Marketplace release 47 | - [x] Integrate Processing 4.3 48 | - [x] Workspace management 49 | - [x] Run and Stop Buttons 50 | - [x] On-hover documentation 51 | - [x] Syntax highlights 52 | - [x] Sketch Errors 53 | - [x] Run Processing sketches headless 54 | - [x] Run Multiple Sketches 55 | - [x] Workspace Management (Sketch and Sketchbook) 56 | - [x] Go-to-definitions 57 | - [x] Go-to-References 58 | - [x] Code Completion 59 | 60 | 61 | ## Limitations 62 | - The LSP features work cross-platform, but sketch execution is currently limited to **Windows** due to the specific [symlink](https://doc.rust-lang.org/std/os/unix/fs/fn.symlink.html) setup for the Windows distribution. 63 | - The large size of the extension is due to custom dependencies downloaded via the `dependency.bat` script. The [dependency.zip](https://github.com/diyaayay/processing-language-server-extension/releases/download/v1.2/dependency.zip) contains the Processing distribution for Windows, which is extracted into `jre/deps`. 64 | - This limitation is due to the extension's JSON-RPC architecture, which enables client-server communication for IntelliSense features. A layered structure was established to connect the protocol, server, client, extension, and Processing distribution, using symlinks to package Processing and execute sketches from the sketch runner. 65 | - Future development could use similar setups for other platforms to expand compatibility. 66 | 67 | ## Future Goals 68 | The following objectives are prioritized for future contributions to the project: 69 | - Establish a comprehensive testing strategy with full feature tests. 70 | - Extend Processing's AST and AST-supported features. 71 | - Bundling of Processing distribution by creating symlinks for multiple platforms (current bundling approach). 72 | - Transition from the custom bundling of Processing with symlink creation for multiple platforms to a modular approach by shipping the core and pre-processor as Maven packages(WIP), and offering a comprehensive CLI interface for Processing and its Language Server, significantly reducing the extension size. 73 | - Enhance the extension's interface to provide an intuitive experience with labeled icons, separate run/stop toggle buttons for each sketch, and other improvements. Drawing inspiration from Antiboredom's [p5.vscode](https://github.com/antiboredom/p5.vscode). 74 | 75 | While VSCode remains one of the most widely used code editors, extending support for this LSP to other editors, including JetBrains, Eclipse, and CodeMirror, can also be a future goal. 76 | 77 | ## Relevant Commits and Issues 78 | - All feature commits are here: [merged code](#what-code-got-merged-in) 79 | - [LSP Release for Processing 4 Beyond the Current Workaround](https://github.com/diyaayay/processing-language-server-extension/issues/1) 80 | - [Licensing discussion](https://github.com/diyaayay/processing-language-server-extension/issues/1) 81 | - [Dynamic sketchpaths](https://github.com/diyaayay/processing-language-server-extension/commit/80e13ed678c8eb464fcccd9b2136fbbd543c25ba) 82 | 83 | ## Important Links 84 | - [Project Link](https://github.com/diyaayay/processing-language-server-extension/) 85 | - [Monthly Updates](https://github.com/processing/pr05-grant/tree/main/2024_NewBeginnings/monthly-reports) 86 | - Marketplace Release: [Download](https://marketplace.visualstudio.com/items?itemName=DiyaSolanki.processing-language-server-extension) 87 | - [GitHub release](https://github.com/diyaayay/processing-language-server-extension/releases/tag/v0.0.1) 88 | - Dependency release: [Processing bundle](https://github.com/diyaayay/processing-language-server-extension/releases/tag/v1.2) -------------------------------------------------------------------------------- /2025_BuildingBridges/final-reports/pr05_2025_final_report_Vaivaswat_Dubey.md: -------------------------------------------------------------------------------- 1 | # Visual Regression Testing Framework for Processing 2 | 3 | **Author:** Vaivaswat Dubey 4 | **Duration:** July – October 2025 5 | 6 | ## Description 7 | Over the course of four months (July–October 2024), I focused on creating a comprehensive visual regression testing framework for the Processing environment that enables automated detection of rendering changes across different platforms and code modifications. 8 | 9 | Visual regression testing is critical for Processing libraries and core development, as even minor code changes can introduce subtle visual artifacts that are difficult to detect manually. 10 | This project delivers a complete, dependency-free testing solution that integrates seamlessly with Processing’s existing Java-based ecosystem and Gradle build system. 11 | 12 | The framework implements a custom pixel-matching algorithm inspired by the proven pixelmatch library, adapted entirely in Java to avoid external dependencies. It provides automated baseline management, platform-specific screenshot comparison, intelligent difference detection, and comprehensive test reporting through JUnit 5 integration. 13 | 14 | 15 | 16 | ## Implementation 17 | 18 | ### Core Infrastructure Components 19 | 20 | #### ImageComparator.java 21 | - Custom pixel-matching algorithm implementing pixelmatch-inspired comparison logic 22 | - Sophisticated cluster analysis to distinguish significant changes from rendering artifacts 23 | - Configurable thresholds: 24 | - MAX_SIDE = 400px 25 | - MIN_CLUSTER_SIZE = 4 26 | - MAX_TOTAL_DIFF_PIXELS = 40 27 | - RGBA color distance calculation using Euclidean distance: 28 | sqrt((dr² + dg² + db² + da²)) / 255.0 29 | - Intelligent filtering of line shifts and anti-aliasing differences 30 | - Automatic diff image generation highlighting differences in red 31 | - Post-processing to identify and filter insignificant pixel clusters 32 | 33 | #### VisualTestRunner.java 34 | - Orchestrates the complete test lifecycle from sketch execution to result reporting 35 | - Platform-specific baseline management with automatic naming (-darwin, -linux, -win32) 36 | - First-run baseline creation with intelligent detection 37 | - Automatic diff image generation and storage on test failures 38 | - Hierarchical screenshot organization in __screenshots__/ directories 39 | - ComparisonResult objects containing pass/fail status, mismatch ratios, and detailed metrics 40 | 41 | #### SketchRunner.java 42 | - Executes Processing sketches in controlled, isolated test environments 43 | - Extends PApplet for native sketch execution 44 | - Critical pixelDensity(1) setting ensuring cross-platform consistency (solves Retina/HiDPI issues) 45 | - Single-frame capture after setup() and one draw() call 46 | - Automatic window cleanup with proper exit() handling 47 | - Configurable render wait times for complex sketches 48 | - Thread synchronization to ensure complete rendering before capture 49 | 50 | --- 51 | 52 | ### Supporting Classes 53 | 54 | - ProcessingSketch Interface – Defines standardized setup() and draw() methods for test sketches 55 | - TestConfig – Configuration management for width, height, background color, comparison threshold, and render timing 56 | - ComparisonResult – Encapsulates test results including pass/fail status, mismatch ratio, and diff image data 57 | - ComparisonDetails – Detailed comparison metrics including total pixels compared, mismatched pixels, and cluster information 58 | - BaselineManager – Utility class for bulk baseline image updates and management 59 | - TestExecutor – Simplifies single-test execution from command line or scripts 60 | 61 | --- 62 | 63 | ### JUnit 5 Integration 64 | 65 | #### VisualTest.java (Base Class) 66 | - Provides assertVisualMatch() helper method for test assertions 67 | - Handles first-run baseline creation using Assumptions.assumeTrue() 68 | - Integrates JUnit assertions for proper test reporting 69 | - Tagged with @Tag("visual") for test filtering and organization 70 | - Supports custom test configurations per test method 71 | 72 | #### Test Suites 73 | - ShapesTestSuite: Organizes shape drawing tests using @Suite and @SelectPackages 74 | - TypographyTestSuite: Groups text rendering and font tests 75 | - Suite-level configuration with @SuiteDisplayName and @IncludeTags 76 | - Enables targeted test execution: 77 | 78 | 79 | ## Comprehensive Test Coverage 80 | 81 | ### Shape Drawing Tests (~17 tests) 82 | - Polyline rendering (open and closed paths) 83 | - Contour drawing (single and multiple nested contours) 84 | - Triangle fans and triangle strips 85 | - Quad strips with proper vertex ordering 86 | - Catmull-Rom spline curves using curveVertex() 87 | - Cubic Bezier curves with bezierVertex() 88 | - Quadratic Bezier curves with quadraticVertex() 89 | - Point and line primitives 90 | - Render mode variations (POINTS, LINES, TRIANGLES, QUADS) 91 | 92 | ### Typography Tests (~20+ tests) 93 | - Dynamic font loading with createFont() 94 | - Parameterized text alignment tests (9 combinations using @ParameterizedTest) 95 | - Horizontal: LEFT, CENTER, RIGHT 96 | - Vertical: TOP, CENTER, BOTTOM, BASELINE 97 | - Multi-line text rendering with manual line breaks 98 | - Text sizing and scaling 99 | - Text leading (line spacing) adjustments 100 | - Text width measurement validation 101 | - Complex rendering scenarios (rotation, transparency, color fills) 102 | - PFont-specific method tests (textAscent(), textDescent()) 103 | 104 | ### Shape-Modes Tests (~15 tests) 105 | - Covers all four shape modes — CORNERS, CORNER, CENTER, and RADIUS — across ellipse, arc, and rect primitives. 106 | - Validates correct placement and scaling of shapes under different coordinate interpretations. 107 | - Includes quadrant-based drawing to verify shape alignment across positive and negative coordinate spaces. 108 | - Tests negative width and height cases to confirm expected flipping and mirroring behavior. 109 | - Ensures ellipseMode, arc, and rectMode produce consistent results across modes. 110 | - Uses fill and stroke variations to confirm rendering consistency and anti-aliasing stability. 111 | - Employs single-frame deterministic rendering for reproducible visual baselines. 112 | 113 | 114 | ## Technical Challenges and Decisions Taken 115 | 116 | ### Challenge 1: Headless Processing Execution 117 | **Problem:** PApplet is fundamentally designed for interactive, windowed execution. Running sketches "headlessly" for automated testing required solving several issues: 118 | - PApplet's lifecycle expects user interaction and continuous draw loops 119 | - No built-in mechanism for single-frame capture 120 | - Thread management complexity when running multiple tests sequentially 121 | 122 | **Solution:** Developed an execution strategy in SketchRunner: 123 | - Override settings() to configure size and pixel density before window creation 124 | - Use noLoop() in setup() to prevent continuous draw calls 125 | - Call redraw() exactly once to execute a single draw cycle 126 | - Implement thread synchronization with CountDownLatch to wait for render completion 127 | - Force window cleanup with surface.setVisible(false) and exit() 128 | 129 | --- 130 | 131 | ### Challenge 2: Binary Development Investigation (August) 132 | 133 | **Problem:** Explored creating standalone executables to eliminate Node.js dependencies and reduce contributor setup friction. 134 | 135 | **Approaches Investigated:** 136 | 137 | | Tool | Verdict | Reason | 138 | |------|----------|--------| 139 | | Nexe | Impractical | ~40-minute builds, large binaries | 140 | | Esbuild | Partial | Required Node.js runtime | 141 | | pkg | Deprecated | Platform build failures | 142 | | Deno | Incompatible | Ecosystem mismatch | 143 | 144 | **Decision:** Abandon binary compilation approaches in favor of native Kotlin implementation. The complexity and limitations of JavaScript-to-binary tools outweighed their potential benefits. 145 | 146 | 147 | 148 | ### Challenge 3: NPM to Kotlin Migration 149 | **Problem:** The initial architecture planned to use the NPM pixelmatch library as a shared component between Processing and p5.js, but this introduced significant complexity. 150 | 151 | **Solution:** Complete reimplementation of the comparison algorithm in Java/Kotlin: 152 | - Analyzed pixelmatch source code to understand core algorithm 153 | - Implemented Euclidean distance color comparison 154 | - Added cluster analysis for filtering insignificant differences 155 | - Developed line-shift detection for anti-aliasing tolerance 156 | - Created platform-specific threshold management 157 | 158 | **Impact:** 159 | - Zero external dependencies for Processing visual tests 160 | - Native integration with Gradle build system 161 | - Faster execution without Node.js overhead 162 | - Simplified contributor setup (no NPM installation required) 163 | - Full control over algorithm tuning for Processing-specific needs 164 | 165 | --- 166 | 167 | ## Architecture Evolution 168 | The project's architecture underwent significant evolution based on practical challenges and community feedback: 169 | 170 | ### **July: Initial Shared Component Vision** 171 | **Plan:** 172 | NPM package shared between Processing and p5.js 173 | 174 | **Architecture:** 175 | - Core comparison engine as an NPM module 176 | - Platform-specific adapters (Java for Processing, JS for p5.js) 177 | - Gradle–NPM integration via [gradle-npm-plugin](https://github.com/node-gradle/gradle-node-plugin) 178 | 179 | 180 | 181 | ### **August: Binary Investigation Phase** 182 | **Exploration:** 183 | Standalone executables to eliminate Node.js dependency 184 | 185 | **Tools Evaluated:** 186 | - Nexe 187 | - pkg 188 | - esbuild 189 | - Deno 190 | 191 | **Outcome:** 192 | All approaches proved unviable due to: 193 | - Excessive build times and large binary sizes 194 | - Platform-specific build failures 195 | - Maintenance concerns with deprecated tools 196 | - Fundamental architectural mismatches 197 | 198 | 199 | 200 | ### **September–October: Native Implementation** 201 | **Decision:** 202 | Complete Kotlin/Java implementation for Processing 203 | 204 | **Benefits:** 205 | - Zero external dependencies 206 | - Native Gradle integration 207 | - Faster execution 208 | - Simplified contributor setup 209 | - Full algorithm control 210 | 211 | **Architecture:** 212 | - Self-contained comparison engine in Java 213 | - JUnit 5-based test framework 214 | - Gradle-native build configuration 215 | - Platform-specific baseline management 216 | 217 | 218 | 219 | This evolution demonstrates the importance of **practical validation over theoretical architecture**. 220 | The final native implementation provides a **superior developer experience** compared to the initial cross-platform shared component vision. 221 | 222 | 223 | **Benefits:** 224 | - Zero external dependencies 225 | - Native Gradle integration 226 | - Faster execution 227 | - Simplified contributor setup 228 | - Full algorithm control 229 | 230 | --- 231 | 232 | ## Key Insights and Lessons Learned 233 | 234 | 1. **Contributor Friction is Critical** 235 | Technical solutions must be evaluated not just on capabilities but on ease of contribution. The native Java implementation requires only JDK installation (already required for Processing development), while the NPM approach would have required Node.js, NPM, and cross-ecosystem knowledge. Lesson: Minimize barrier to entry over technical elegance. 236 | 237 | 2. **Headless GUI Applications Are Non-Trivial** 238 | Running PApplet non-interactively required deep understanding of its lifecycle, threading model, and window management. The "just run it without a window" approach doesn't work. Lesson: GUI frameworks make assumptions about interactivity that must be explicitly managed in automated testing. 239 | 240 | 3. **Binary Compilation Has Hidden Costs** 241 | The investigation into Nexe, pkg, and other tools revealed that JavaScript-to-binary compilation is far from mature. Build times, binary sizes, platform inconsistencies, and deprecated toolchains make this approach impractical for most projects. Lesson: Sometimes the "boring" solution (native implementation) is the best solution. 242 | 243 | 4. **Threshold Tuning is an Art** 244 | Finding the right balance for MAX_TOTAL_DIFF_PIXELS, MIN_CLUSTER_SIZE, and color tolerance required extensive experimentation with real sketches. Too sensitive = false positives, too lenient = missing real bugs. Lesson: Default thresholds should work for most cases, but allow per-test customization. 245 | 246 | 5. **Documentation in Code** 247 | The typography test challenges (white canvas problem) highlighted the need for comprehensive setup examples. Creating font explicitly, setting fill in draw(), using BASELINE alignment - these aren't obvious from Processing documentation. Lesson: Test code serves as documentation; make it exemplary. 248 | 249 | --- 250 | 251 | ## Future Goals and Enhancements 252 | 253 | ### Expand Test Coverage 254 | - 3D rendering tests (box, sphere, lights, camera) 255 | - Image loading and manipulation tests 256 | - Pixel array manipulation tests 257 | - Shader tests 258 | - Video and camera tests 259 | 260 | ### Performance Optimization 261 | - Parallel test execution avoiding window conflicts 262 | - Algorithm optimization for large images 263 | - Smart baseline caching 264 | 265 | ### Enhanced Reporting 266 | - HTML diff report generation showing side-by-side comparisons 267 | - Threshold suggestion based on historical test data 268 | - Visual summary dashboard for test suites 269 | 270 | --- 271 | 272 | ## Important Links and References 273 | 274 | - Current Implementation: [processing4/core/test](https://github.com/Vaivaswat2244/processing4/tree/more-visual-tests/core/test/processing/visual) 275 | - Monthly Reports: [Project Timeline](https://github.com/processing/pr05-grant/tree/main/2025_BuildingBridges/monthly-reports) 276 | - NPM Package: [visual-regression-engine (archived, superseded by native implementation)](https://www.npmjs.com/package/visual-regression-engine) 277 | 278 | --- 279 | 280 | ## Conclusion 281 | 282 | The Visual Regression Testing framework for Processing represents a significant enhancement to the Processing development ecosystem. 283 | By providing automated, reliable visual testing with zero external dependencies, the framework enables developers to catch rendering regressions before they reach users. 284 | 285 | The journey from the initial NPM-based architecture to the final native implementation demonstrates the importance of practical validation and community feedback in open source development. 286 | While the path included detours through binary compilation and cross-platform integration challenges, the final solution provides a robust, maintainable foundation for Processing visual testing. 287 | -------------------------------------------------------------------------------- /2025_BuildingBridges/final-reports/pr05_2025_final_report_Claire_Peng/README.md: -------------------------------------------------------------------------------- 1 | # pr05 2025 Typescript Migration 2 | 3 | This is the project appendix for the **2025 pr05 Grant: Incremental Typescript Migration of the p5.js Web Editor**. 4 | 5 | ## Additional Reading: 6 | 7 | - [All PRs related to the pr05 grant can be found with the `pr05` tag](https://github.com/processing/p5.js-web-editor/issues?q=state%3Aclosed%20label%3A%22pr05%20Grant%20Projects%22) 8 | - [pr05 grant](https://github.com/processing/pr05-grant/wiki/2025-pr05-Program-Page) 9 | - [pr05 grant: Typescript Migration Project](https://github.com/processing/pr05-grant/wiki/2025-pr05-Project-List#incremental-typescript-migration-for-the-p5js-editor) 10 | - [Intro to the Incremental Typescript Migration for the p5.js Web Editor](https://medium.com/@clairepeng94/intro-to-the-incremental-typescript-migration-for-the-p5-js-web-editor-2ffbc305c6a9) 11 | 12 | ## Table of Contents 13 | 14 | - [Project Outline](#project-outline) 15 | - [Context](#project-context) 16 | - [Proposed Approach](#proposed-approach) 17 | - [Project Timeline](#project-timeline) 18 | - [All PRs](#all-prs) 19 | - [Summary of Outcome](#outcome) 20 | - [Key Decisions](#key-decisions) 21 | - [Summary of TS Configurations](#configuration-summary) 22 | - [Migration Tutorial](#migration-tutorial) 23 | - [Index of Migration Examples](#examples-index) 24 | - [Next Step Issues](#next-step-issues) 25 | - [Additional Reading](#additional-reading) 26 | 27 | ## Project Outline: 28 | 29 | ### Project Context: 30 | 31 |
32 | View details 33 | 34 | - Grant period of 200h over July 1st - October 31, 2025 35 | - 116k lines of code within the repo, with about 30% test coverage 36 | - Many older technologies, tied to Node 16 (bumped to Node 18 in August) 37 | 38 |
39 | 40 | ### Proposed Approach: 41 | 42 | > To set up the repo so that beginning contributors can pick up migration work as a "Good first issue" 43 | 44 |
45 | Set up all config to be both TS & JS compatible 46 | 47 | - Not possible to migrate entire repo within 200h 48 | - Migration must also be file-by-file, so all config needs to be flexible to work with legacy JS code 49 | - Key technologies: 50 | - Node 51 | - React 52 | - Express 53 | - MongoDB 54 | - Jest 55 | - Babel 56 | - Webpack 57 | - ESLint 58 | 59 |
60 | 61 |
62 | Set up typechecking tooling & automation 63 | 64 | 65 | - Global typecheck command for both the `/server` and `/client` folders 66 | - Isolated typecheck command for each the `/server` and `/client` folders 67 | - Automated typecheck on pre-commit hook 68 | - Automated typecheck on CICD tests 69 | 70 |
71 | 72 |
73 | 74 | Migrate an instance of each "kind" of file a developer might encounter to provide example code to follow 75 | 76 | 77 | - Client: 78 | - React file 79 | - Plain TS file (eg. `/utils`) 80 | - Server: 81 | - Mongoose Model --> `User` 82 | - Mongoose Controller --> `User.Controller` 83 | - Express Routes 84 | - Plain TS file (eg. `/utils`) 85 | - Examples of writing tests for the above 86 | - Example of extending namespaced types through `.d.ts` files 87 | 88 |
89 | 90 | ### Project Timeline: 91 | 92 | Click to view details by month 93 | 94 |
95 | July: 96 | 97 | - Set up TS dependencies & configuration on the `root` 98 | - `tsconfig.json` 99 | - `tsconfig.base.json` 100 | - `.babelrc` 101 | - `.eslintrc` 102 | - `package.json > jest.config` 103 | - `webpack/` 104 | - Set up TS dependencies & configuration in `/client` 105 | - `/client/tsconfig.json` 106 | - `@types/...` for `/client` dependencies (eg. React 16) 107 | - Set up `typecheck` and `typecheck:client` commands in `package.json` 108 | - Migrate `/client/utils` 109 | 110 |
111 | 112 |
113 | August: 114 | 115 | - Migrate `/client/common` with co-located types & tests 116 | - Migrate `/client/components` with co-located types & tests 117 | 118 |
119 | 120 |
121 | September: 122 | 123 | - Set up TS dependencies & configuration in `/server` 124 | - `nodemon.json` 125 | - `webpack/` 126 | - `/server/tsconfig.json` 127 | - `@types/...` for `/server` dependencies 128 | - Set up `/server/types` to store all `/server` types (eg Mongoose v8) 129 | - Migrate `/server` components related to emailing and `nodemailer` 130 | - Migrate `/server/routes` 131 | - Migrate `/server/models/User` with tests 132 | - Migrate `/server/models/ApiKeys` with tests 133 | - Start migrating `/server/controllers/User` 134 | 135 |
136 | 137 |
138 | October: 139 | 140 | - Complete migration for `/server/controllers/User` 141 | - Added tests to all the methods & re-organised by sub-domain (eg. auth management) & added JSDocs for each controller method. 142 | - Instantiate `.d.ts` files for `express` and `jest-express` to create a custome `Express` namespace `Request.User` definition 143 | - Set up `/common/types` folder to 144 | - Migrate root redux files: `/client/store`, `/client/reducers` & `/client/persistState` 145 | - Migrate user preferences redux: `/client/IDE/reducers/preferences` & `/client/IDE/actions/preferences` 146 | - Migrate `/client/modules/Legal` 147 | - Migrate `/client/modules/About` 148 | - Migrate most of `/client/modules/User` (with exception of files related to `Collections`) 149 | - Clean up of previous work & documentation 150 | 151 |
152 | 153 | ## All PRs: 154 | 155 | Click to view all PRs for this project: 156 | 157 |
158 | Click to view all PRs for this project: 159 | 160 | 161 | - [pr05 Typescript #1: Set up TS dependencies in the root & client](https://github.com/processing/p5.js-web-editor/pull/3533) 162 | - [pr05 Typescript #2: automatically resolve imports of ts files & migrate instance of client/utils file](https://github.com/processing/p5.js-web-editor/pull/3540) 163 | - [pr05 Typescript #3: Migrate client/utils folder](https://github.com/processing/p5.js-web-editor/pull/3553) 164 | - [pr05 Typescript #4: migrate instance of react file](https://github.com/processing/p5.js-web-editor/pull/3554) 165 | - [pr05 Typescript Migration #5: Migrate client/common folder](https://github.com/processing/p5.js-web-editor/pull/3565) 166 | - [pr05 Typescript Migration #6: Initial docs on typescript migration](https://github.com/processing/p5.js-web-editor/pull/3581) 167 | - [pr05 Typescript Migration #7: migrate client components](https://github.com/processing/p5.js-web-editor/pull/3619) 168 | - [pr05 Typescript #8: migrate client/components/Menubar/MenubarSubmenu](https://github.com/processing/p5.js-web-editor/pull/3623) 169 | - [pr05 Typescript Migration #10: Setup Server TS Dependencies & migrate instance of server file](https://github.com/processing/p5.js-web-editor/pull/3636) 170 | - [pr05 Typescript Migration #11: Migrate server/routes folder](https://github.com/processing/p5.js-web-editor/pull/3643) 171 | - [pr05 Typescript Migration 12: Migrate server files related to emailing/nodemailer](https://github.com/processing/p5.js-web-editor/pull/3658) 172 | - [pr05 Typescript Migration #13: Migrate the User Model (& API Key schema)](https://github.com/processing/p5.js-web-editor/pull/3672) 173 | - [pr05 Typescript Migration #14: Migrate User Controller](https://github.com/processing/p5.js-web-editor/pull/3681) 174 | - [pr05 Typescript Migration #15: Redux base files & migrate user preferences redux system](https://github.com/processing/p5.js-web-editor/pull/3683) 175 | - [pr05 Typescript Migration #16: Fix build error webpack example config](https://github.com/processing/p5.js-web-editor/pull/3696) 176 | - [pr05 Typescript Migration #17: Add typecheck to ci](https://github.com/processing/p5.js-web-editor/pull/3699) 177 | - [Pr05 Typescript Migration #18: Migrate client/modules/About & client/modules/Legal](https://github.com/processing/p5.js-web-editor/pull/3702) 178 | - [Pr05 Typescript Migration #19: client/common/icons](https://github.com/processing/p5.js-web-editor/pull/3703) 179 | - [Pr05 Typescript Migration #20: Refine server/types & client/reducer types](https://github.com/processing/p5.js-web-editor/pull/3704) 180 | - [Pr05 Typescript Migration #21: client/modules/User](https://github.com/processing/p5.js-web-editor/pull/3705) 181 | - [pr05 Typescript Migration - Followon Project: Swagger / OpenApi documentation & UI for server routes](https://github.com/processing/p5.js-web-editor/pull/3706) 182 | 183 |
184 | 185 | ## Outcome: 186 | 187 | Click to view details & relevant files per outcome-item 188 | 189 |
190 | All TS-related configuration for dependencies is completed 191 | 192 | - `tsconfig.json` 193 | - `tsconfig.base.json` 194 | - `.babelrc` 195 | - `.eslintrc` 196 | - `package.json > jest.config` 197 | - `webpack/` 198 | - `nodemon.json` 199 | - `/client/tsconfig.json` 200 | - `/server/tsconfig.json` 201 |
202 | 203 |
204 | Typecheck commands have been set up & enabled on automated checks locally and on CI 205 | 206 | - `npm run typecheck` to check both the `/server` and `/client` folders 207 | - `npm run typecheck:server` to check only the `/server` folder 208 | - `npm run typecheck:client` to check only the `/client` folder 209 | - `package.json > husky` configured to run `npm run typecheck` during `pre-commit` check 210 | - `/github/workflows/test.yml` configured to run `npm run typecheck` during `test` on GHA 211 | 212 |
213 | 214 |
215 | The following files have been migrated, covering at least once instance of each 'kind' of file encountered on the repo: 216 | 217 | - [x] `/common/types` 218 | - [.] `/client/`: 219 | - [x] `/client/utils/` 220 | - [x] `/client/common/` 221 | - [x] `/client/components/` 222 | - [x] `/client/store` 223 | - [x] `/client/reducers` 224 | - [x] `/client/persistState` 225 | - [x] `/client/IDE/reducers/preferences` 226 | - [x]`/client/IDE/actions/preferences` 227 | - [x] `/client/modules/User/` 228 | - [x] `/client/modules/About/` 229 | - [x] `/client/modules/Legal/` 230 | - [x] `/client/custom.d.ts` 231 | - [.] `/server`: 232 | - [.] `/server/controllers` 233 | - [x] `/server/controllers/user.controller/` 234 | - [.] `/server/models/` 235 | - [x] `/server/models/user` 236 | - [x] `/server/models/apiKey` 237 | - [x] `/server/routes/` 238 | - [x] `/server/middleware/` 239 | - [x] `/server/types/` 240 | - [x] `/server/views/` 241 | - [.] `/server/utils/` 242 | - [x] `/server/utils/generateFileSystemSafeName` 243 | - [x] `/server/utils/mail` 244 | - [x] `/server/utils/renderMjml` 245 | 246 |
247 | 248 | ## Key Decisions: 249 | 250 |
251 | 252 | Typescript is configured to be used for type checking only, not for transpiling 253 | 254 | 255 | ```shell 256 | npm run tsc --noEmit 257 | ``` 258 | 259 | - The repo is too large to convert entirely to TS in one go, so we need to flexibly support both TS and JS files 260 | - We have existing legacy runtime build tools (Babel & Webpack) for the existing JS, so this was the easiest way to support both our intended direction and legacy codebase 261 | - Our configured automated type checking prevents PRs from being merged if they fail, so we're working towards a Typescript system without disrupting current build. 262 | - Once the repo has been migrated, we can update to full transpilation with `npm run tsc` 263 | 264 |
265 | 266 |
267 | 268 |
269 | 270 | Prefer `named` exports over `default` exports where possible 271 | 272 | 273 | ❌ DON'T: 274 | 275 | ```ts 276 | // calculator.ts 277 | export default function multiply(a: number, b: number): numer { 278 | return a * b; 279 | } 280 | 281 | // main.ts 282 | import calculate from "./calculator"; // the default function is renamed and confusing for next dev to read 283 | 284 | console.log(calculate(2, 2)); // is this 2*2? 2+2? 2^2? something else? 285 | ``` 286 | 287 | ✅ DO: 288 | 289 | ```ts 290 | // calculator.ts 291 | export function multiply(a: number, b: number): numer { 292 | return a * b; 293 | } 294 | 295 | // main.ts 296 | import { multiply } from "./calculator"; 297 | 298 | console.log(multiply(2, 2)); 299 | ``` 300 | 301 | - Default exports allow devs to rename the import anything they want, and inaccurate misnaming will cause confusion 302 | - Renaming default export also makes it more difficult to perform a global search on the function 303 | 304 |
305 | 306 |
307 | 308 | Prefer `interface` over `type` 309 | 310 | 311 | ```tsx 312 | // ✅ interface 313 | interface ButtonProps { 314 | label: string 315 | disabled?: boolean 316 | onClick: () => void 317 | } 318 | const Button = ({ label, disabled, onClick }: ButtonProps) => (...) 319 | 320 | // ❌ type 321 | type ButtonProps = { 322 | label: string 323 | disabled?: boolean 324 | onClick: () => void 325 | } 326 | 327 | // ✅ keep type for primitives 328 | type UserId = string; 329 | ``` 330 | 331 | - Clearer semantic meaning when describing structured objects vs. primitives 332 | - This is not strictly enforced in linting rules, but aims to align with [Google Typescript styling](https://google.github.io/styleguide/tsguide.html#prefer-interfaces). Please see attached link for details. 333 | 334 |
335 | 336 |
337 | 338 |
339 | 340 | If possible, add unit tests prior to the component that you are migrating prior to performing any migration 341 | 342 | 343 | - This protects the app from unintended regression & also helps demonstrate to reviewers that you understand the intent of a component/file you are migrating 344 | - Tests also help document how the component is intended to work. 345 | 346 |
347 | 348 |
349 | 350 | If a component does not have an existing unit test, keep refactors/logic changes outside of Typescript Migration PRs 351 | 357 | 358 |
359 | 360 |
361 | 362 | Add JSDocs & specify the `Express` `RequestHandler` generics to `/server` controller methods in the following example: 363 | 364 | 365 | ```ts 366 | // see server/controllers/user.controller/userPreferences.ts 367 | 368 | /** 369 | * - Method: `PUT` 370 | * - Endpoint: `/preferences` 371 | * - Authenticated: `true` 372 | * - Id: `UserController.updatePreferences` 373 | * 374 | * Description: 375 | * - Update user preferences, such as AppTheme 376 | */ 377 | export const updatePreferences: RequestHandler< 378 | {}, // request params type if there are any, extending RouteParam 379 | UpdatePreferencesResponseBody, // response body type 380 | UpdatePreferencesRequestBody // request body type 381 | // query params type if there are any 382 | > = async (req, res) => { 383 | // ... rest of code 384 | }; 385 | ``` 386 | 387 | - JSDocs will reduce cognitive load for other developers and enable them to see the controllers details on hover. 388 | - Add the JSDocs to any `client` controllers that use these endpoints. 389 | 390 | hover on a function with jsdocs 391 | 392 |
393 | 394 |
395 | 396 |
397 | 398 | Best-effort & evolving-precision principle towards defining types 399 | 400 | 401 | - Because we are adding types into a legacy project originally written without types, there are sometimes inconsistencies between component contracts, or misalignments between type shapes or name conventions that make adding in types difficult. 402 | - As such make a best effort at being as precise as possible with context clues, but when in doubt, selecting a broader type (eg. `string` instead of an `enum`) is valid and we can update to be stricter as the migration continues. 403 | 404 |
405 | 406 |
407 | 408 |
409 | 410 | `/server` types live in `/server/types`. `/client` types are co-located (currently). Shared types between the `/server` and `/client` live in `/common/types`. 411 | 412 | 413 | - `/server` types are in `/types` folder 414 | - These types should serves the source-of-truth for the rest of the app. 415 | - Pull any types that are relevant to share with the `./client` into the `./common/types` folder 416 | - `/client` types are co-located for now 417 | - Components have their own types so they will get very numerous and messy 418 | - However this is open to proposals for change 419 | 420 |
421 | 422 | ## Configuration Summary: 423 | 424 |
425 | 426 | Relevant PRs 427 | 428 | 429 | - [pr05 Typescript #1: Set up TS dependencies in the root & client](https://github.com/processing/p5.js-web-editor/pull/3533) 430 | - [pr05 Typescript #2: automatically resolve imports of ts files & migrate instance of client/utils file](https://github.com/processing/p5.js-web-editor/pull/3540) 431 | - [pr05 Typescript Migration #10: Setup Server TS Dependencies & migrate instance of server file](https://github.com/processing/p5.js-web-editor/pull/3636) 432 | - [pr05 Typescript Migration #15: Redux base files & migrate user preferences redux system](https://github.com/processing/p5.js-web-editor/pull/3683) 433 | - [pr05 Typescript Migration #16: Fix build error webpack example config](https://github.com/processing/p5.js-web-editor/pull/3696) 434 | - [pr05 Typescript Migration #17: Add typecheck to ci](https://github.com/processing/p5.js-web-editor/pull/3699) 435 | 436 |
437 | 438 | ## Migration Tutorial: 439 | 440 | [Video Guide - Migrating the `client/modules/User/pages/AccountView`](https://youtu.be/y84SVy7lAgg) 441 | 442 | 443 |
444 | 445 | Text Guide 446 | 447 | TO DO 448 |
449 | 450 | 451 | ## Examples Index: 452 | 453 | ### Client Files: 454 | 455 |
456 | React component 457 | 458 | - [IconButton](https://github.com/processing/p5.js-web-editor/blob/develop/client/common/IconButton.tsx) 459 | - [IconButton test](https://github.com/processing/p5.js-web-editor/blob/develop/client/common/IconButton.test.tsx) 460 | 461 |
462 | 463 |
464 | Normal TS file 465 | 466 | - [formatDate](https://github.com/processing/p5.js-web-editor/blob/develop/client/utils/formatDate.ts) 467 | - [formateDate test](https://github.com/processing/p5.js-web-editor/blob/develop/client/utils/formatDate.test.ts) 468 | 469 |
470 | 471 |
472 | Hook 473 | 474 | - [usePrevious](https://github.com/processing/p5.js-web-editor/blob/develop/client/common/usePrevious.ts) 475 | - [usePrevious test](https://github.com/processing/p5.js-web-editor/blob/develop/client/common/usePrevious.test.ts) 476 | 477 |
478 | 479 |
480 | Redux 481 | 482 | - To be updated pending open PR 483 | 484 |
485 | 486 |
487 | Custom type declaration file 488 | 489 | - [custom.d.ts](https://github.com/processing/p5.js-web-editor/blob/develop/client/custom.d.ts) 490 | - Use this to extend any client namespaced types (eg. Window) or file extensions (.svg, .mp3) 491 | 492 |
493 | 494 | ### Server Files: 495 | 496 |
497 | Types are extracted into /server/types in a barrel structure 498 | 499 | - [types/userPreferences](https://github.com/processing/p5.js-web-editor/blob/develop/server/types/userPreferences.ts) 500 | - [types/index](https://github.com/processing/p5.js-web-editor/blob/develop/server/types/index.ts) 501 | - All server types are exported here 502 | 503 |
504 | 505 |
506 | Routes 507 | 508 | - [routes/user.routes.ts](https://github.com/processing/p5.js-web-editor/blob/develop/server/routes/user.routes.ts) 509 | - Format the routes by 'subdomain' if the file is particularly large 510 | - Add the method & path in code comments for easier global search 511 | 512 |
513 | 514 |
515 | Model 516 | 517 | - [models/user](https://github.com/processing/p5.js-web-editor/blob/develop/server/models/user.ts) 518 | - [models/user test](https://github.com/processing/p5.js-web-editor/blob/develop/server/models/__test__/user.ts) 519 | 520 |
521 | 522 |
523 | Controller 524 | 525 | - [controllers/user.controller > index file](https://github.com/processing/p5.js-web-editor/blob/develop/server/controllers/user.controller/index.ts) 526 | - Note that controllers should be organised in barrel format with an index file. 527 | - [controllers/user.controller > updateSettings method](https://github.com/processing/p5.js-web-editor/blob/develop/server/controllers/user.controller/userPreferences.ts) 528 | - [controllers/user.controller > updateSettings method test](https://github.com/processing/p5.js-web-editor/blob/develop/server/controllers/user.controller/__tests__/userPreferences.test.ts) 529 | 530 |
531 | 532 |
533 | Custom type declaration for a namespace for a dependency 534 | 535 | - [types/express](https://github.com/processing/p5.js-web-editor/blob/develop/server/types/express/index.d.ts) 536 | - Extend the `express` namespace `Request.User` with a custom user type. 537 | - [types/jest-express](https://github.com/processing/p5.js-web-editor/blob/develop/server/types/jest-express/index.d.ts) 538 | - Parallel change on `jest-express` to extend `Request.User` in the test environment 539 | 540 |
541 | 542 | ### Common Files: 543 | 544 |
545 | Types shared across the server to client are exported in /common/types 546 | 547 | - [types/index](https://github.com/processing/p5.js-web-editor/blob/develop/common/types/index.ts) 548 | - Note that for some APIs (eg. user) we don't want to export **all** types, we want to specifically name the types to share with the client (eg. not `UserDocument`) 549 | 550 |
551 | 552 | ## Next Step Issues: 553 | 554 |
555 | Migrate the rest of the repo to Typescript 556 | 557 | - Prioritise `/server` if possible 558 | - Testing is mandatory for `/server` migration 559 | - For `/client` files, if the component does not hit the `/server` api, it is ready for migration. 560 | - If it does it is recommended to wait for the relevant `/server` system to be migrated first (eg. Projects, Collections) 561 | 562 |
563 | 564 |
565 | Add end-to-end tests to cover the core user flows 566 | 567 | - Writing a suite of e2e tests to cover core user flows would enable security against regression with fewer test files necessary 568 | - This can be done with Playwrite or Cypress & can be integrated into CICD pipelines. 569 | - Examples of core user flows include: 570 | - `User can log in` 571 | - `User can save a sketch & see it in their sketches` 572 | - To be discussed further with the contributor community. 573 | 574 |
575 | 576 |
577 | Create a OpenAPI/Swagger documentation microsite for the server APIs 578 | 579 | proof of concept of a swagger documentation microsite 580 | 581 | - [Swagger.ui](https://swagger.io/tools/swagger-ui/) 582 | 583 | - This would enable new contributors to have a centralised location to look for APIs 584 | - Also aligns with current API documentation standards as of 2025 585 | - Current API documentation lives [here](https://github.com/processing/p5.js-web-editor/blob/develop/contributor_docs/public_api.md) and does not contain all the APIs 586 | - [Proof of concept PR here](https://github.com/processing/p5.js-web-editor/pull/3706) 587 | - I have written out the User related APIs by hand, but ideally we would have automation to check that this document is accurate, or to autogenerate the document, or autogenerate types & routes from the document. 588 | 589 |
590 | --------------------------------------------------------------------------------