Exploring the World of Music Technology with AI

Studying Targets

By engaged on this mission, we stand to achieve new technical expertise and an understanding of how AI algorithms could be applied to construct modern functions. By the top of this mission, we are going to:

1. Achieve an understanding of how synthetic intelligence is employed in creating music. We’ll be taught the elemental ideas and strategies used to coach AI fashions for music composition.
2. Discover ways to acquire and put together related musical knowledge for AI mannequin coaching. We’ll uncover how you can collect .mp3 recordsdata and convert them into MIDI recordsdata, using instruments corresponding to Spotify’s Fundamental Pitch.
3. We may even perceive the steps concerned in constructing an AI mannequin for music era. Additional, we are going to be taught in regards to the mannequin structure appropriate for this activity and its relevance and acquire hands-on expertise in coaching the mannequin, together with figuring out the variety of epochs and batch dimension.
4. We’ll spend time to find strategies to judge the efficiency of the educated mannequin. Then we are going to discover ways to analyze metrics and assess the standard of generated music items to gauge the mannequin’s effectiveness and establish areas for enchancment.
5. Lastly, we are going to discover the method of utilizing the educated AI mannequin to generate new musical compositions.

This text was printed as part of the Knowledge Science Blogathon.

Venture Description

The aim of this mission is to discover the intriguing area of music era utilizing AI. We goal to research how synthetic intelligence strategies create distinctive musical items. By leveraging machine studying algorithms, our goal is to coach an AI mannequin able to producing melodies and harmonies throughout varied musical genres.

The mission’s focus is on gathering a various vary of musical knowledge, particularly .mp3 recordsdata, which can function the muse for coaching the AI mannequin. These recordsdata will bear preprocessing to transform them into MIDI format utilizing specialised instruments like Spotify’s Fundamental Pitch. This conversion is important as MIDI recordsdata present a structured illustration of musical parts that the AI mannequin can simply interpret.

The following section includes constructing the AI mannequin tailor-made for music era. Prepare the mannequin utilizing the ready MIDI knowledge, aiming to seize underlying patterns and buildings current within the music.

Conduct the efficiency analysis to evaluate the mannequin’s proficiency. It will contain producing music samples and assessing their high quality to refine the method and improve the mannequin’s capability to provide artistic music.

The ultimate consequence of this mission would be the capability to generate authentic compositions utilizing the educated AI mannequin. These compositions could be additional refined by means of post-processing strategies to complement their musicality and coherence.

Drawback Assertion

The mission endeavours to sort out the problem of restricted accessibility to music composition instruments. Conventional strategies of music creation could be laborious and demand specialised information. Furthermore, producing contemporary and distinct musical ideas can pose a formidable problem. The goal of this mission is to make use of synthetic intelligence to avoid these obstacles and supply a seamless resolution for music era, even for non-musicians. Via the event of an AI mannequin with the potential to compose melodies and harmonies, the mission goals to democratize the method of music creation, empowering musicians, hobbyists, and novices to unleash their artistic potential and craft distinctive compositions with ease.

A Temporary Historical past of Music Technology Utilizing AI

The story of AI in making tunes goes again to the Nineteen Fifties, with the Illiac Suite for String Quartet being the primary tune made with a pc’s assist. Nonetheless, it’s solely in the previous couple of years that AI has actually began to shine on this space. Right this moment, AI could make tunes of many sorts, from classical to pop, and even make tunes that duplicate the model of well-known musicians.

The present state of AI in making tunes may be very advance within the current instances. Just lately, Meta has introduced out a brand new AI-powered tune maker known as MusicGen. MusicGen, made on a robust Transformer mannequin, can guess and make music components in the same solution to how language fashions guess the subsequent letters in a sentence. It makes use of an audio tokenizer known as EnCodec to interrupt down audio knowledge into smaller components for straightforward processing.

One of many particular options of MusicGen is its capability to deal with each textual content descriptions and music cues on the similar time, leading to a clean mixture of creative expression. Utilizing a giant dataset of 20,000 hours of allowed music, ensuring its capability to create tunes that join with listeners. Additional, firms like OpenAI have made AI fashions like MuseNet and Jukin Media’s Jukin Composer that may make tunes in a variety of types and kinds. Furthermore, AI can now make tunes which are nearly the identical as tunes made by people, making it a robust device within the music world.

Moral Concerns

Discussing the moral elements of AI-generated music is essential when exploring this area. One pertinent space of concern includes potential copyright and mental property infringements. Prepare AI fashions on intensive musical datasets, which might end in generated compositions bearing similarities to current works. It’s important to respect copyright legal guidelines and attribute authentic artists appropriately to uphold truthful practices.

Furthermore, the appearance of AI-generated music might disrupt the music business, posing challenges for musicians in search of recognition in a panorama inundated with AI compositions. Putting a stability between using AI as a artistic device and safeguarding the creative individuality of human musicians is an important consideration.

Knowledge Assortment & Preparation

For the aim of this mission, we are going to attempt to generate some authentic instrumental music utilizing AI. Personally, I’m a giant fan of famend instrumental music channels like Fluidified, MusicLabChill, and FilFar on YouTube, which have glorious tracks for all types of temper. Taking inspiration from these channels, we are going to try and generate music on comparable traces, which we are going to lastly share on YouTube.

To assemble the mandatory knowledge for our mission, we concentrate on sourcing the related .mp3 recordsdata that align with our desired musical model. Via intensive exploration of on-line platforms and web sites, we uncover authorized and freely obtainable instrumental music tracks. These tracks function invaluable property for our dataset, encompassing a various assortment of melodies and harmonies to complement the coaching means of our mannequin.

As soon as we have now efficiently acquired the specified .mp3 recordsdata, we proceed to remodel them into MIDI recordsdata. MIDI recordsdata symbolize musical compositions in a digital format, enabling environment friendly evaluation and era by our fashions. For this conversion, we depend on the sensible and user-friendly performance offered by Spotify’s Fundamental Pitch.

With the help of Spotify’s Fundamental Pitch, we add the acquired .mp3 recordsdata, initiating the transformation course of. The device harnesses superior algorithms to decipher the audio content material, extracting essential musical parts corresponding to notes and buildings to generate corresponding MIDI recordsdata. These MIDI recordsdata function the cornerstone of our music era fashions, empowering us to govern and produce contemporary, modern compositions.

Mannequin Structure

To develop our music era mannequin, we make the most of a specialised structure tailor-made particularly for this function. The chosen structure includes two LSTM (Lengthy Quick-Time period Reminiscence) layers, every consisting of 256 items. LSTM, a kind of recurrent neural community (RNN), excels in dealing with sequential knowledge, making it a superb selection for producing music with its inherent temporal traits.

The primary LSTM layer processes enter sequences with a hard and fast size of 100, as decided by the sequence_length variable. By returning sequences, this layer successfully preserves the temporal relationships current within the musical knowledge. To stop overfitting and enhance the mannequin’s adaptability to new knowledge, a dropout layer with a dropout charge of 0.3 is included.

The second LSTM layer, which doesn’t return sequences, receives the outputs from the earlier layer and additional learns intricate patterns throughout the music. Lastly, a dense layer with a softmax activation operate generates output chances for the next be aware.

Constructing the Mannequin

Having established our mannequin structure, let’s dive straight into constructing the identical. We’ll break down the code into sections and clarify every half for the reader’s sake.

We begin by importing the mandatory libraries that present helpful functionalities for our mission. Along with the same old libraries required for normal ops, we might be utilizing tensorflow for deep studying, and music21 for music manipulation.

import numpy as np
import os
import tensorflow as tf
from tensorflow.keras.fashions import Sequential
from tensorflow.keras.layers import LSTM, Dropout, Dense
from tensorflow.keras.utils import to_categorical
from music21 import converter, instrument, stream, be aware, chord
from google.colab import recordsdata

Loading and Processing MIDI Recordsdata

Subsequent, we outline the listing the place our MIDI recordsdata are positioned. The code then goes by means of every file within the listing, extracts the notes and chords, and shops them for additional processing. The ‘converter’ module from the music21 library is used to parse the MIDI recordsdata and retrieve the musical parts. As an experiment, we are going to first use only one MIDI file to coach the mannequin after which evaluate the consequence through the use of 5 MIDI recordsdata for coaching.

# Listing containing the MIDI recordsdata
midi_dir = "/content material/Midi Recordsdata"

notes = []

# Course of every MIDI file within the listing
for filename in os.listdir(midi_dir):
    if filename.endswith(".midi"):
        file = converter.parse(os.path.be part of(midi_dir, filename))

        # Discover all of the notes and chords within the MIDI file
        strive:
            # If the MIDI file has instrument components
            s2 = file.components.stream()
            notes_to_parse = s2[0].recurse()
        besides:
            # If the MIDI file solely has notes (
            # no chords or instrument components)
            notes_to_parse = file.flat.notes

        # Extract pitch and period data from notes and chords
        for aspect in notes_to_parse:
            if isinstance(aspect, be aware.Observe):
                notes.append(str(aspect.pitch))
            elif isinstance(aspect, chord.Chord):
                notes.append('.'.be part of(str(n) for n in 
                aspect.normalOrder))

# Print the variety of notes and a few instance notes
print("Whole notes:", len(notes))
print("Instance notes:", notes[:10])

Mapping Notes to Integers

To transform the notes into numerical sequences that our mannequin can course of, we create a dictionary that maps every distinctive be aware or chord to a corresponding integer. This step permits us to symbolize the musical parts in a numerical format.

# Create a dictionary to map distinctive notes to integers
unique_notes = sorted(set(notes))
note_to_int = {be aware: i for i, be aware in 
enumerate(unique_notes)}

Producing Enter and Output Sequences

So as to practice our mannequin, we have to create enter and output sequences. That is carried out by sliding a fixed-length window over the record of notes. The enter sequence consists of the previous notes and the output sequence is the subsequent be aware. These sequences are saved in separate lists.

# Convert the notes to numerical sequences
sequence_length = 100  # Size of every enter sequence
input_sequences = []
output_sequences = []

# Generate enter/output sequences
for i in vary(0, len(notes) - sequence_length, 1):
    # Extract the enter sequence
    input_sequence = notes[i:i + sequence_length]
    input_sequences.append([note_to_int[note] for 
    be aware in input_sequence])

    # Extract the output sequence
    output_sequence = notes[i + sequence_length]
    output_sequences.append(note_to_int[output_sequence])

Reshaping and Normalizing Enter Sequences

Earlier than feeding the enter sequences to our mannequin, we reshape them to match the anticipated enter form of the LSTM layer. Moreover, we normalize the sequences by dividing them by the overall variety of distinctive notes. This step ensures that the enter values fall inside an appropriate vary for the mannequin to be taught successfully.

# Reshape and normalize the enter sequences
num_sequences = len(input_sequences)
num_unique_notes = len(unique_notes)

# Reshape the enter sequences
X = np.reshape(input_sequences, (num_sequences, sequence_length, 1))
# Normalize the enter sequences
X = X / float(num_unique_notes)

One-Scorching Encoding Output Sequences

The output sequences representing the subsequent be aware to foretell will convert right into a one-hot encoded format. This encoding permits the mannequin to grasp the chance distribution of the subsequent be aware among the many obtainable notes.

# One-hot encode the output sequences
y = to_categorical(output_sequences)

Defining the RNN Mannequin

We outline our RNN (Recurrent Neural Community) mannequin utilizing the Sequential class from the tensorflow.keras.fashions module. The mannequin consists of two LSTM (Lengthy Quick-Time period Reminiscence) layers, adopted by a dropout layer to stop overfitting. The final layer is a Dense layer with a softmax activation operate to output the possibilities of every be aware.

# Outline the RNN mannequin
mannequin = Sequential()
mannequin.add(LSTM(256, input_shape=(X.form[1], X.form[2]), 
return_sequences=True))
mannequin.add(Dropout(0.3))
mannequin.add(LSTM(256))
mannequin.add(Dense(y.form[1], activation='softmax'))

Compiling and Coaching the Mannequin

We compile the mannequin by specifying the loss operate and optimizer. We then proceed to coach the mannequin on the enter sequences (X) and output sequences (y) for a particular variety of epochs and with a given batch dimension.

# Compile the mannequin
mannequin.compile(loss="categorical_crossentropy", optimizer="adam")

# Step 4: Prepare the mannequin
mannequin.match(X, y, batch_size=64, epochs=100)

Music Technology

As soon as we practice the mannequin, we will generate new music sequences. We outline a operate named generate_music that takes three inputs: the educated mannequin, seed_sequence, and size. It makes use of the mannequin to foretell the subsequent be aware within the sequence based mostly on the earlier notes and repeats this course of to generate the specified size of music.

To begin, we create a replica of the seed_sequence to stop any modifications to the unique sequence. This seed_sequence serves because the preliminary level for producing the music.

We then enter a loop that runs size instances. Inside every iteration, carry out the next steps:

1. Convert the generated_sequence right into a numpy array.
2. Reshape the input_sequence by including an additional dimension to match the anticipated enter form of the mannequin.
3. Normalize the input_sequence by dividing it by the overall variety of distinctive notes. This ensures that the values fall inside an appropriate vary for the mannequin to work successfully.

After normalizing the input_sequence, use the mannequin to foretell the possibilities of the subsequent be aware. The mannequin.predict technique takes the input_sequence as enter and returns the expected chances.

To pick out the subsequent be aware, the np.random.selection operate is used, which randomly picks an index based mostly on the possibilities obtained. This randomness introduces range and unpredictability into the generated music.

The chosen index represents the brand new be aware, which is appended to the generated_sequence. The generated_sequence is then up to date by eradicating the primary aspect to take care of the specified size. As soon as the loop completes, the generated_sequence is returned, representing the newly generated music.

The seed_sequence and the specified generated_length should be set to generate the music. The seed_sequence needs to be a legitimate enter sequence that the mannequin has been educated on, and the generated_length determines the variety of notes the generated music ought to comprise.

# Generate new music
def generate_music(mannequin, seed_sequence, size):
    generated_sequence = seed_sequence.copy()

    for _ in vary(size):
        input_sequence = np.array(generated_sequence)
        input_sequence = np.reshape(input_sequence, (1, len(input_sequence), 1))
        input_sequence = input_sequence / float(num_unique_notes)  # Normalize enter sequence

        predictions = mannequin.predict(input_sequence)[0]
        new_note = np.random.selection(vary(len(predictions)), p=predictions)
        generated_sequence.append(new_note)
        generated_sequence = generated_sequence[1:]

    return generated_sequence

# Set the seed sequence and size of the generated music
seed_sequence = input_sequences[0]   # Change with your individual seed sequence
generated_length = 100  # Change with the specified size of the generated music

generated_music = generate_music(mannequin, seed_sequence, generated_length)
generated_music

# Output of the above code
[1928,
 1916,
 1959,
 1964,
 1948,
 1928,
 1190,
 873,
 1965,
 1946,
 1928,
 1970,
 1947,
 1946,
 1964,
 1948,
 1022,
 1945,
 1916,
 1653,
 873,
 873,
 1960,
 1946,
 1959,
 1942,
 1348,
 1960,
 1961,
 1971,
 1966,
 1927,
 705,
 1054,
 150,
 1935,
 864,
 1932,
 1936,
 1763,
 1978,
 1949,
 1946,
 351,
 1926,
 357,
 363,
 864,
 1965,
 357,
 1928,
 1949,
 351,
 1928,
 1949,
 1662,
 1352,
 1034,
 1021,
 977,
 150,
 325,
 1916,
 1960,
 363,
 943,
 1949,
 553,
 1917,
 1962,
 1917,
 1916,
 1947,
 1021,
 1021,
 1051,
 1648,
 873,
 977,
 1959,
 1927,
 1959,
 1947,
 434,
 1949,
 553,
 360,
 1916,
 1190,
 1022,
 1348,
 1051,
 325,
 1965,
 1051,
 1917,
 1917,
 407,
 1948,
 1051]

Put up-Processing

The generated output, as seen, is a sequence of integers representing the notes or chords in our generated music. So as to hearken to the generated output, we must convert this again into music by reversing the mapping we created earlier to get the unique notes/chords. To do that, we are going to firstly create a dictionary known as int_to_note, the place the integers are the keys and the corresponding notes are the values.

Subsequent, we create a stream known as output_stream to retailer the generated notes and chords. This stream acts as a container to carry the musical parts that can represent the generated music.

We then iterate by means of every aspect within the generated_music sequence. Every aspect is a quantity representing a be aware or a chord. We use the int_to_note dictionary to transform the quantity again to its authentic be aware or chord string illustration.

If the sample is a chord, which could be recognized by the presence of a dot or being a digit, we break up the sample string into particular person notes. For every be aware, we create a be aware.Observe object, assign it a piano instrument, and add it to the notes record. Lastly, we create a chord.Chord object from the notes record, representing the chord, and append it to the output_stream.

If the sample is a single be aware, we create a be aware.Observe object for that be aware, assign it a piano instrument, and add it on to the output_stream.

As soon as all of the patterns within the generated_music sequence have been processed, we write the output_stream to a MIDI file named ‘generated_music.mid’. Lastly, we obtain the generated music file from Colab utilizing the recordsdata.obtain operate.

# Reverse the mapping from notes to integers
int_to_note = {i: be aware for be aware, i in note_to_int.objects()}

# Create a stream to carry the generated notes/chords
output_stream = stream.Stream()

# Convert the output from the mannequin into notes/chords
for sample in generated_music:
    # sample is a quantity, so we convert it again to a be aware/chord string
    sample = int_to_note[pattern]

    # If the sample is a chord
    if ('.' in sample) or sample.isdigit():
        notes_in_chord = sample.break up('.')
        notes = []
        for current_note in notes_in_chord:
            new_note = be aware.Observe(int(current_note))
            new_note.storedInstrument = instrument.Piano()
            notes.append(new_note)
        new_chord = chord.Chord(notes)
        output_stream.append(new_chord)
    # If the sample is a be aware
    else:
        new_note = be aware.Observe(sample)
        new_note.storedInstrument = instrument.Piano()
        output_stream.append(new_note)

# Write the stream to a MIDI file
output_stream.write('midi', fp='generated_music.mid')

# Obtain the generated music file from Colab
recordsdata.obtain('generated_music.mid')

Ultimate output

Now, it’s time to hearken to the result of our AI-generated music. You could find the hyperlink to hearken to the music beneath.

To be sincere, the preliminary consequence might sound like somebody with restricted expertise enjoying musical devices. That is primarily as a result of we educated our mannequin utilizing solely a single MIDI file. Nonetheless, we will improve the standard of the music by repeating the method and coaching our mannequin on a bigger dataset. On this case, we are going to practice our mannequin utilizing 5 MIDI recordsdata, all of which might be instrumental music of the same model.

The distinction within the high quality of the music generated from the expanded dataset is sort of outstanding. It clearly demonstrates that coaching the mannequin on a extra various vary of MIDI recordsdata results in important enhancements within the generated music. This emphasizes the significance of accelerating the scale and number of the coaching dataset to attain higher musical outcomes.

Limitations

Although we managed to generate music utilizing a complicated mannequin, however there are specific limitations to scaling such a system.

1. Restricted Dataset: The standard and variety of the generated music depend upon the variability and dimension of the dataset used for coaching. A restricted dataset can prohibit the vary of musical concepts and types our mannequin can be taught from.
2. Creativity Hole: Though AI-generated music can produce spectacular outcomes, it lacks the inherent creativity and emotional depth that human composers deliver to their compositions. The music generated by AI might sound robotic or miss the refined nuances that make music really fascinating.
3. Knowledge Dependency: Affect the generated music by the enter MIDI recordsdata used for coaching. If the coaching dataset has biases or particular patterns, the generated music might exhibit comparable biases or patterns, limiting its originality.
4. Computational Necessities: Coaching and producing music utilizing AI fashions could be computationally costly and time-consuming. It requires highly effective {hardware} and environment friendly algorithms to coach complicated fashions and generate music in an affordable timeframe.
5. Subjective Analysis: Assessing the standard and creative worth of AI-generated music could be subjective. Totally different folks might have totally different opinions on the aesthetics and emotional influence of the music, making it difficult to determine common analysis requirements.

Conclusion

On this mission, we launched into the fascinating journey of producing music utilizing AI. Our aim was to discover the capabilities of AI in music composition and unleash its potential in creating distinctive musical items. Via the implementation of AI fashions and deep studying strategies, we efficiently generated music that intently resembled the model of the enter MIDI recordsdata. The mission showcased the flexibility of AI to help and encourage within the artistic means of music composition.

Key Takeaways

Listed below are among the key takeaways from this mission:

1. We realized that AI can function a helpful assistant within the artistic course of, providing new views and concepts for musicians and composers.
2. The standard and variety of the coaching dataset significantly affect the output of AI-generated music. Curating a well-rounded and diversified dataset is essential to reaching extra authentic and various compositions.
3. Whereas AI-generated music reveals promise, it can’t substitute the creative and emotional depth introduced by human composers. The optimum method is to leverage AI as a collaborative device that enhances human creativity.
4. Exploring AI-generated music raises essential moral concerns, corresponding to copyright and mental property rights. It’s important to respect these rights and foster a wholesome and supportive setting for each AI and human artists.
5. This mission bolstered the importance of steady studying within the area of AI-generated music. Staying up to date with developments and embracing new strategies permits us to push the boundaries of musical expression and innovation.

Continuously Requested Questions

The media proven on this article isn’t owned by Analytics Vidhya and is used on the Creator’s discretion.

Associated