What if AI could help us better understand art? What if it could help us all better recognize popular art movements?

Oracle Cloud Infrastructure AI Vision is a fully managed AI service that performs image recognition tasks like classifying images, detecting objects & detecting faces. It can be used to leverage pre-trained models or create custom models for specific use cases. As you can imagine, recognizing art style is definitely a specific use case.

In this article, I will attempt to train my own model to perform custom image classification.

Sounds like a complex task ? I wanted to see it by myself!

Before diving into Oracle Cloud, we need data, a lot of data, and even better than data, some sort of pre-labelled data, ie samples of paintings with their corresponding art style. This will allow the AI model to learn and be applicable to images it never saw before (if you ever heard the expression supervised learning, this is about it).

For that purpose, I used the Artbench dataset : https://github.com/liaopeiyuan/artbench

Artbench contains more than 50 000 images and 11 different art styles.

Now let's move to OCI.

We first start by going into the Analytics & AI menu, Vision, and in the Custom Models section => Projects.

This is where I create my first project :

Completing this form ...

Once the status of the project becomes active, I click on it and have the following view :

I click on "Create Model" button. From there I define my target model as an image classification model.

I will create a new dataset - in the OCI sense- as I haven't done so already. This redirects me to the OCI Data Labeling service.

When I try to upload the files directly, it seems the uploading interface limits me to 100 files, which is not enough for me obviously.

I decide to upload some of the 5000 expressionist paintings into the appropriate object storage bucket (I created a few Object Storage buckets, each containing 5000 images of a specific art style )

My 2nd attempt to upload the files from the Object storage interface was also unsuccessful even though I could upload more than the 100 files like in the Data Labeling interface.

As soon as I tried to upload more than 500 files it started to crash, so I went to the old-fashioned solution : Cyberduck. A colleague from Oracle,
Birzu Alexandru-Adrian, uploaded a great post if you want to use Cyberduck to connect to OCI Object storage.

Once uploaded, we can finally create our data set and hopefully apply bulk labeling to each set of images coming from a specific bucket :

OMG I realize my data is there, but it is not labelled. I would have expected an automated bulk labeling feature as the task takes a ridiculous amount of time to do manually (it seems all major cloud vendors are lacking such feature AFAIK !), but anyway !

Instead of writing my own script to batch label the image, I will reuse Oracle's Data Science AI scripts on github (I choose Java over Python):

https://github.com/oracle-samples/oci-data-science-ai-samples/tree/master/data_labeling_examples/bulk_labeling_java

Before using that script, I realize I have to reorganize my training data. Why ? Because I also expected a dataset to be able to include multiple data sources, ie, multiple object storage buckets. Unfortunately, this is not possible today.

There you are, all the data must be uploaded into a single object storage bucket ! Which means I need to find a way to differentiate each painting's art style to make the labelling as automated as possible.

I achieve that by adding a prefix that will reflect each image's style. Flexible Renamer seems like a sensible option to start with.

As a result each file is prefixed with a style and a white space like this :

Now that all my files (more or less) are uploaded into the OCI's object storage bucket, I can re-create the OCI dataset.

This is done in OCI Data Labelling Service :

I'm almost ready run the bulk labelling script locally using the FIRST_REGEX_MATCH" labeling algorithm. Please note that the script did not work "out of the box", I had to modify the different urls and dataset OCIDs (addresses) but also remove the DCONFIG parameters while modifying the config.properties file. Last but not least I also changed the regex to extract the first word from the filename.

I changed :

java -DCONFIG_FILE_PATH='~/.oci/config' -DCONFIG_PROFILE=DEFAULT -DDLS_DP_URL=https://dlsprod-dp.us-ashburn-1.oci.oraclecloud.com -DTHREAD_COUNT=20 -DDATASET_ID=ocid1.compartment.oc1..aaaaaaaawob4faujxaqxqzrb555b44wxxrfkcpapjxwp4s4hwjthu46idr5a -DLABELING_ALGORITHM=FIRST_REGEX_MATCH -DFIRST_MATCH_REGEX_PATTERN=^abc* -DLABELS=cat,dog -cp libs/bulklabelutility-v2.jar com.oracle.datalabelingservicesamples.scripts.SingleLabelDatasetBulkLabelingScript

into

java -DDLS_DP_URL=https://dlsprod-dp.eu-frankfurt-1.oci.oraclecloud.com -DTHREAD_COUNT=30 -DDATASET_ID=ocid1.datalabelingdataset.oc1.eu-frankfurt-1.amaaaaaazgiyodqawmwcg3dxgjwjd24pwxpekw5wcaab5z34zbm5wnzcjoba -DLABELING_ALGORITHM=FIRST_REGEX_MATCH -DFIRST_MATCH_REGEX_PATTERN=(^\S*) -DLABELS=baroque,expressionism,impressionism,realism,renaissance,romanticism,surrealism -cp libs/bulklabelutility-v2.jar com.oracle.datalabelingservicesamples.scripts.SingleLabelDatasetBulkLabelingScript

Please allow me to save you a few hours of debugging here. If you try running this script and get Label is null for record RecordSummary this is probably because either your regex is not correct, or because the labels you created on the OCI Data Labelling Service console do not exactly match the labels you try to use in the commands (do not try to put less or more ! It must be a 1-1 relationship)

As a result I got 7000 images labeled in 2 minutes:

This gets confirmed in the OCI Console (check the Labeled number) :

Side story : The first time I tried this uploaded 26K files and saw only 10000 labeled files, I understood that happened because I just hit the service limit (I believe this can be increased upon a service limit request from the console ):

During my 2nd try, I didn't use any window software to upload the file but rather the java program provided:

java -DOBJECT_STORAGE_URL=https://objectstorage.eu-frankfurt-1.oraclecloud.com -DOBJECT_STORAGE_BUCKET_NAME=WH-Dataset4Vision-ArtBench-UberFolderMixed01 -DOBJECT_STORAGE_NAMESPACE=fr1wb0c6sbky -DDATASET_DIRECTORY_PATH=C:\Users\walid\Downloads\artbench-10-imagefolder-split\train\miniUber -cp libs/bulklabelutility-v1.jar com.oracle.datalabelingservicesamples.scripts.UploadToObjectStorageScript

Now that I have a data set called Name: WH-Art-ArtBench-256256-dataset05mixed with around 1000 pictures of each style, I'm reading to create & train my model

Let's go back to OCI Vision to create and train a new model :

For the purpose of this blog post, I choose "quick mode" although it would have been wiser to keep "recommended mode".

(a couple of hours later)

Once trained, we obtain a model that has a certain ability to predict the art style of an image. Interestingly, we don't need to upload any further test data, as OCI Vision took care of it by extracting a part of what we have uploaded in the dataset.

The model obtained in the output has a certain precision & recall. Precision is the division between True Positives (for a certain label classification) divided by the total of True Positives + False Positives. As you can imagine, the more we get close to 1.0 the better, meaning all predicted positives are truly positives.

Recall is using the same numerator [TP] while the denominator is [TPs + False Negatives], it answers the question: What proportion of actual positives was identified correctly? (source: https://developers.google.com/machine-learning/crash-course/classification/precision-and-recall ). This is the same as what we know as Sensitivity in Biostatistics.

The F1 Score is a combination of both used to have a single classification quality indicator :

$$F1 = 2(Precision Recall) / (Precision+ Recall)$$

The higher the F1 score, the better.

Our F1 score of 61.09 should do the job for the demo. By going through the image catalog, I'm not surprised we don't get a better accuracy as many of the paintings could have been way better classified. This is where educated humans are much needed :-) otherwise it will just generate average results.

Let's try our model with test data that has never been used for the training.

We obtain a great style recognition with this Gauguin Painting.

Our AI even nails it (95% confidence) when it comes to recognizing Dali's masterpieces :

The results are very promising given that

• we didn't use the full recommended training mode

• we only used a small subset of the initial ArtBench data set

• and that we didn't use any human-in-the-loop to better label the training data

So we can easily imagine how powerful training custom models can be using OCI AI Vision.

I hope you enjoyed reading this article and that you'll start training your own AI models on what matters to you !

Walid Hajeri

Disclaimer : Views are my own , none of the ideas expressed in this post are shared, supported, or endorsed in any manner by my current employer.

Cover picture : Photo by Eduardo Romero from Pexels: pexels.com/photo/human-face-painting-on-wal..