learning_to_draw_02

July 2024

A single image which combines a collection of smaller images of lines and shapes

Samples of the the training data used

learning_to_draw_02: AI-Powered G-Code Generation for Machine Drawing

Introduction

Building on my previous work with machine drawing, I’m excited to share my latest project: learning_to_draw_02. This initiative explores the intersection of artificial intelligence and creative expression through the lens of G-code generation for 2D plotters and 3D printers.

Project Overview

The core concept of learning_to_draw_02 is to harness the power of state-of-the-art open-source AI models to create a Large Language Model (LLM) capable of generating G-code from images. This approach offers a novel way to translate visual information into machine instructions for drawing or printing.

Key Components

  1. Dataset Creation: I’ve developed a procedural method to generate paired datasets of images and their corresponding G-code instructions.

  2. AI Model Training: The project utilizes a vision transformer neural network, trained to take an image as input and produce the G-code required to draw that image.

  3. Development Process: I leveraged AI assistance in the coding process, using ChatGPT for quick suggestions and Claude.ai for more in-depth debugging and refinement.

Technical Implementation

The heart of the project lies in creating a multi-modal image-to-text model using the Hugging Face ecosystem. This involves:

  • Defining a vision transformer model
  • Training on the custom dataset of images and G-code pairs
  • Utilizing the HUGGINGFACE_API_KEY for seamless integration

Future Directions

As the project evolves, I’m considering additional features such as:

  • A G-code to PNG turtle image renderer
  • Potential blockchain integration for image verification and provenance

Conclusion

learning_to_draw_02 represents an exciting step forward in the realm of AI-assisted creative tools. By bridging the gap between visual input and machine instructions, this project opens up new possibilities for artists, makers, and researchers interested in computational creativity.

I look forward to sharing more updates as the project progresses and exploring the potential applications of this technology in various fields.


Thanks Claude.


Occasional posts on topics including AI, deep learning, and generative art.

© 2024 Matthew Hollings