An Introduction to Collaborating with AI in Creative Work

Fortune’s Path contributor and artist Ted Noser takes you through his journey of using AI to compose a song and make a video to demonstrate some best practices for using AI in creative work.

Playing With AI Will Make You Smarter

Everyone who works in technology and creative fields should be playing with AI right now because AI can make you smarter.

You’ll make some terrible stuff, but in the process, you’ll refine your aesthetic sense through trial and error. AI can remove a lot of the dirty work of creation so creators can all focus on becoming better curators. When creating, it’s better to have good taste than to be original, whether you’re writing a song or software.

Outsource Effectively

When I started this project, I didn’t know much about how to use AI to make music. I felt anxious and confused. So I had two tasks I needed help with: attaining mental clarity and picking AI music tools.

I asked ChatGPT to help me understand my mood and find musical tools to express it. It fed me words like “energized, relaxed, somber” asking me to pick the ones I identified with. I responded earnestly, yet still couldn’t pin down how I felt.

I didn’t want to to write lyrics for the song, so I decided it would be better to use abstract words to describe my feelings, as I had the intention of making abstract, instrumental music, rather than imagistic music like a soundtrack.

After divulging my anxieties to ChatGPT, it offered some advice and a few tools: 

Choose the right tools

Your choice of platforms and software can determine the difference between dull and derivative or strong and original work. 

Most AI content is horrible. It looks like things we’ve seen before, lacks a compelling focus, or is such an awful mishmash that it’s numbing.  

My favorite AI artists use more than one tool to make their work. They curate a toolbox that helps them express their ideas and explore new avenues of creation. I’ll take you step by step as I create a song and accompanying video to show you how to use AI as a collaborator. 

Being a good creator with AI means you know how to choose and use the right tools.

I have quite a bit of experience making ambient music with my musical project Law Office. When experimenting with new processes, it’s wise to use familiar forms of art to experiment with. But I am inherently hesitant to embrace an AI tool's conception of what could make for “soothing” music. After all, one man’s “soothing music” is another’s “elevator jazz.” So I took ChatGPTs advice to explore free AI programs for creative functions but was reluctant to follow its creative direction.

When I tried to follow ChatGPT advice about tools, right away I ran into problems.

The link to the music-making tool it had suggested, Amper, wouldn’t load. I later realized that Amper had never been a free-access AI music-making tool. Shame on you ChatGPT! So I gave up on ChatGPT’s recommendations and turned to good old-fashioned word of mouth.

A friend had recently told me about Google’s MusicFX. Clicking it, the site booted up just fine. Score one for social skills and Google. Minus one for OpenAI.

Using Google’s MusicFX to make “soothing music”

Available through Google’s free AI test kitchen, MusicFX creates original music using suggestions from user-generated text prompts. My effort to make a soothing song with this tool felt predestined, as many things often do when search engines steal our data. When I entered the site, the example prompt matched the words ChatGPT suggested to describe the mood I was hoping to convey in my music:

“Sounds that create a soothing and adventurous atmosphere”

I gave in to algorithmically driven fate and used the example prompt. MusicFX returned a dreamy song that sounded like it was coming from a synthetic harp.

The melody relied heavily on the Lydian mode and a whole tone scale, and had a lovely lo-fi quality. Eager to explore more, I input my first custom prompt:

“A dreamy melody that gently wakes you on a rainy day in the countryside”

Out came a monstrosity. It was an alarm clock tone capable of triggering PTSD. Nothing soothing about it. But, at the very least, I was inspired to use the program more.

After some failed attempts to micromanage the software, it became apparent that this was a program intended to work within very limited circumstances and considerations of music. Musical terminology was not its strong suit, genre matching was. So I decided to keep my prompts vague. Asking the program to generate “a dreamy melody from wind chimes,” I got a clip that sounded vaguely like the band Beach House. The melody was still too “busy,” but I decided I could use the audio as a base to sample later.

Blend Tools For Improved Results

With the audio finished, I returned to ChatGPT to find a visual to accompany my AI-generated sound-thing (it was definitely not a song whatever it was.) It pointed me to Deep Dream, an AI tool that uses text prompts to generate surreal, dreamy visuals. I’d already described the audio as having a similar vibe to Beach House, so I figured that would be a good starting point for the text prompt.

I plugged in “a beach house built on memories.” After waiting about two minutes for my video to render, the result was vaguely interesting as a linguistic interpretation, but boring to watch. Dissatisfied, I knew I would need another tool. So I downloaded the video to manipulate it again later with Genmo AI, an image to video animator.  

When I tossed a screenshot from the Deep Dream video into Genmo AI for further manipulation, Genmo interpreted my screenshot as text.

 It was now my turn to improve on this prompt to get a more intriguing video. I settled on “Small house with a porch on a mushroom floating on a moon in a psychedelic dreamscape.” The result looked like it belonged in a Mellow Mushroom, but seeing as the intended use of this video was to make someone on Instagram stop and watch for more than 5 seconds, I felt it was a better result. 

A common theme in AI art is the uncanny effect that comes from the combination of pseudo realist movement with photorealistic image renders. To chase this effect, I decided to hop over to Edenart, another AI visual tool. Edenart is vastly more fun, interesting and engaging than Deepdream and Genmo, but it does cost money. So I’m breaking from the mold for a $10 month service for a (hopefully) better result. 

Knowing that I enjoyed photorealism, I decided to use the “Generator; vid2vid” feature in Eden, which “remixes” a current video’s visual style by mashing it up with a “style image,” but preserves motion and duration, and uses a picture of a face for a sprinkle of extra-uncanny effect. 

I hope the Catholic church doesn’t condemn me for this, but the first person I could think of was Pope Francis, so I grabbed an image from Google images, making sure to pick a portrait that focused on his face and had minimal background, so the “style” of the image would reflect the pope. 

The audio didn’t have a direct effect on the outcome of this video, but it did influence the prompts I used that led me to the final project.

AI art requires a lot of stumbling. It’s an immature collaborator. It doesn’t work well to express a clear idea you already have in your head. The tools I used take broad direction for interpretation. If you know what you want to see, make your creation yourself. If you need help making something and aren’t too picky with the results, AI can help.

AI Doesn’t mean fire your workers.

Though the piece I made isn't very good, I hope my process demonstrates how much you still have to think to make good content with AI. Creatives, your jobs are safe but you need to get smart about AI to keep up.

Managers, don’t fire your creatives because you believe you can do it better with AI. AI is a tool that can reduce the tedium of certain processes, but at this stage, it can't replace the expertise required to do good work in a creative endeavor.