Skip to content

Week: Mini-Project #2

UPDATE

AI Generated Images from Lyrics to 'Don't Stop Me Now' by QueenClick on image above to watch Midjourney AI-generated video using lyrics to "Don't Stop Me Now" by Queen

Overview

This week we bring together all the many disparate and not-so-disparate skills honed over the past several weeks into Mini-Project #2. Over the past few months, text to image (and now video) generation has seen a dramatic leap forward with the introduction of new large AI models. This also aligns nicely with our own AI research, so we can lend an unusual degree of experience to this rapidly evolving field.

Creating text prompts (prompt engineering) to feed into these text2image models is one of the hottest areas of AI research recently, and we'll explore this further this week. This involves an all-hands-on-deck class project that decomposes nicely into smaller groups: text to image generation by recently released state-of-the-art large DNN models.

GOAL:

Research, search and scrape Twitter for images and prompts based upon the following 3 state-of-the-art text2image deep neural network (DNN) models:

  1. DALL-E 2: @openai
  2. Midjourney: @midjourney
  3. Stable Diffusion: @stablediffusion

Our goal is to use what we've learned about scraping, APIs, NLP and image processing to find tweets with text generated images on Twitter. In particular, we want to collect the following:

  1. the author/organization
  2. text prompt with the associated
  3. generated image
  4. any other relevant information, comments or meta-information

Everyone will be assigned into one of three groups representing each of the main three text2image DNN models listed above. Each group will independently research, scrape and analyze as much data as they can for their assigned DNN model. Then as a group, we will combine, compare, and critique our findings as a unified team.

Readings



Assignments:

Teams:

Social Media DALL-E 2 Midjourney Stable Diffusion
Twitter Freddie Jill Jeremy
Ani Claire Vikas
Reddit Viet Max Devon
Teddy Anav Abbie
Tao

Identify Scrape Targets: * Official Social Media Accounts (e.g. @handles) * Groups/Boards of Fans (e.g. subreddits) * Search Terms (e.g. regular or #hashtags) * Individual Artists * Gallaries (e.g. Lexica.art) * Other sources (e.g. Slack, Discord)

Guidance: * Keep good notes as you go * Give FULL DETAILS we need to scrape BOTH (a) Generated Images and (b) text prompts (+ possible explainations/given context) - Full URL paths/subdirectories - Unique usernames/handles/hashtags - Distinct and effective keywords/search terms - etc * Try to group common patterns into a taxonomy as you to, make notes on what distinctive features you are using to base your classification on

References

Twitter API Ver 2

Discord

Slack

Other Scrapers & Bots

Code Samples

Books