Commons talk:ISA Tool/Image to Concept

Requesting feedback - 4th of October 2022

Helloǃ We have just released a new version of the ISA tool on the Test Server.
We are looking for feedback with regards to the look, feel, and usability of the tool with the machine vision / metadata-to-concept enhancements in place. We deeply appreciate your feedback, since it will help us determine the next steps of the project. Thank you in advanceǃ

1. Please sign-up to be updated about the project
2. This is the link to the beta version of the enhanced tool: https://isa-dev.toolforge.org.
3. To fully test the tool, please try it before (read-only) and after logging in with your usual Wikimedia username.
4. You may use one of the existing campaigns or create one of your own.
5. When logged in, try to make meaningful edits, as the tags you enter will be written to Wikimedia Commons ;)
6. Add your feedback below.
7. If you need to display a screenshot, report on that doc

 Go to the test environment

Notes

Your feedback

Images didn't load after creating new campaign

Created a new campaign; the images didn't load for ca. 24 hours. It worked only after ticking the "update images" box on the campaign edit page. --Beat Estermann (talk) 13:27, 4 October 2022 (UTC)

+1 Beireke1 (talk) 10:06, 14 October 2022 (UTC)

ISA Tool temporarily down

Got an error message as the ISA Tool was temporarily down (see Screenshot in Google Doc). --Beat Estermann (talk) 13:27, 4 October 2022 (UTC)

ISA Tool in maintenance

I faced maintenance when i want to test the tool Agus Damanik (talk) 23:31, 7 October 2022 (UTC)

Error Message

I couldn't save my edit to proceed to another page. Iwuala Lucy (talk) 07:08, 9 October 2022 (UTC)

Error Message: Your Edits have not been saved to Wikimedia Commons

When trying to save my tags (after flagging the prominent element), I am getting the error message: “Oops! Something went wrong, your edits have not been saved to Wikimedia Commons” (see screenshot).

Strangely enough, the “depicts” statements actually got recorded (but not the “prominent” information): [2]

--Beat Estermann (talk) 09:00, 10 October 2022 (UTC)

New Image takes too long to load

When clicking on "Next Image", the associated metadata loaded right away, but the image itself took about four seconds to load - during that time the wrong image was displayed, i.e. there was a mismatch between image and metadata. --Beat Estermann (talk) 09:03, 10 October 2022 (UTC)

After contributing to a campaign, the campaign statistics were not updated right away

After contributing to a campaign (UNESCO campaign), the campaign statistics still showed "0" participants and "0" contributions. The issue at first persisted after reloading the Campaigns Overview page. After reloading 5 minutes later, the issue was resolved and the statistics were displayed correctly. --Beat Estermann (talk) 09:28, 10 October 2022 (UTC)

+1, although it took more than one hour before the updated statistics showed up. Beireke1 (talk) 13:57, 10 October 2022 (UTC)

Overly generic, no exclusion list

One of the problems with the machine tagging is that it produces a bunch of generic tags, rather than Depicts statements, that are ultimately useless for high quality curation of content. ISA campaigns already produce a lot of generic content with novice users, the machine tags increases that. The Commons:Structured data/Computer-aided tagging tool encountered something similar, and without a crowd-curated exclusion list - somewhere to say "this is out of scope for depicts statements" -- you will get 1000s more depicts statement about "sky" or "grass" which does no-one any good: it floods commons search results with not useful photograph, it dilutes the impact of high quality depicts statements, and it creates a bad user experience for newcomers (creating conflict, confrontation or reversion of edits)-- a well thought through, integrated way to stop users from getting a particular suggested tag seems to be really important in the user experience, Sadads (talk) 22:40, 11 October 2022 (UTC)

Hello user:Sadads. Thank you for taking the time to provide detailed feedback and critical thinking. I will mostly note that... there are two rounds of feedback... the one being about the look, feel, and usability of the tool, whilst the second will be about the suggestions themselves. This is round 1 :) The work on the recommandations is still ongoing...

But the issue you raise is clearly an important one. A few weeks ago, I experienced first hand the situation of edit reversion due to tags being high level / generic. It felt like some editors were ready to dismiss generic tags, to favor highly specific ones. But then, I was pointed to that page Commons:Structured data/Modeling/Depiction as being the nearest thing to a consensus. See the « level of details » paragraph. What are your thoughts on that ? Anthere (talk)

Hi @Sadads: . Here is the current deny list User:NavinoEvans/denylist. What do you think ? Anthere (talk) 00:56, 26 October 2022 (UTC)

Sorry for the slow response. As far as I read the conversations from the community, and the guidance -- it should be the things most visible in an image -- redundancy is okay for some of the topics, but we really need to be careful about generic terms (such as themes in images or items that are not clearly the subject of the image). I think having a denylist is a good way to keep newcomers from becoming overzealous, while also allowing experienced users to do more nuanced description. Sadads (talk) 14:11, 3 November 2022 (UTC)

No suggestions

Even after login in I get no suggestions according to the specific picture in some existing campaigns (Leo Wehrli, World heritage, Natural history museum). Surprisingly I get suggestions in the small campaign: Turkey photographs taken on 2020-06-14 --Hadi (talk) 15:43, 17 October 2022 (UTC)

I wonder if this might be due to [3] but in any cases, I see no images at all in the Turkey photographs campaign. I just asked that the campaigns images be reloaded on the test environnement. Thanks Anthere (talk) 23:48, 25 October 2022 (UTC)

Depicts not the same as categories?

I am confused. I thought the Depicts concept was being developed for AI identification, so that, for example, the system could learn that an image with "sky", "clouds", "tree" and "road surface" is a landscape. Whereas of course these four terms are completely useless for categorization (except for an image of a cloudy sky, but then the narrowest category might be "Sunsets in X country (by year)" or something like that). I have uploaded many original photos and am baffled at the purpose of all the extra labor required to fill out the Depicts statements.

If Depicts is supposed to categorize, then the instruction is inadequate. "Name the different elements that you can see in this image" asks the categorizer to treat the image as a set of objects. I don't see how that will advance the goal of making Wikimedia Loves X images more useful to the public. "Street sign" "umbrella" "cafe" and "beach" do not tell us that this is a picture of a rural area near X town in Africa. If the goal is somewhat useful categorization, I would change the instruction to "If you had to describe this photograph in one sentence to someone else, what information would you include? For example, "this is a picture of a gardener digging a vegetable garden outside a house in X village in Y country in Z year."" Downtowngal (talk) 00:13, 22 October 2022 (UTC)

An issue with marking prominent elements

The feature of marking an element as prominent doesn't work on elements that were previously saved. That is if you mark the prominent elements with the flag before saving the elements it works well. But if you list some elements and click on save, you cannot comme back later and mark some of the elements as prominent. Each time I tried, I got the error "Oops! Something went wrong, your edits have not been saved to Wikimedia Commons". Serieminou (talk) 03:06, 25 October 2022 (UTC)

ok. Yes, I just tried and got the same thing. That’s a bug to fix... Anthere (talk) 23:52, 25 October 2022 (UTC)

Draft of guidelines

Given that the current pages are quite long and not easy to find for most editors, I drafted a « shorter » version of Depiction Guidelines, with the hope of have it added to the guidelines collection...

This is a super condensed version of the current pages (Depicts and Modeling), trying to take into account the current perspective rather than the historical expectations. Please give your feedback... that would be very helpful : Commons:Depiction guidelines. Anthere (talk)

See also the discussion at the Village Pump. --Beat Estermann (talk) 14:40, 4 November 2022 (UTC)

@Beat Estermann: @Islahaddow: @Ceslause: @Secretlondon: @OtuNwachinemere: @Onyinyeonuoha: @Magotech: @Bile rene: @Asaf (WMF): @Beireke1: @Mndetatsin: @GeorgHH: @Serieminou: @Downtowngal: @Hadi: @Beireke1: @Sadads: @Iwuala Lucy: @Agus Damanik:

Deny list

Should elements such as Should depicts such as black-and-white photography Q6901463 appear in the suggestions ? Or not ?

See Depiction guidelines for food for thought. Anthere (talk) 08:20, 31 October 2022 (UTC)

Add a help bubble

Next to the suggestions area, to explain where those recommandations come from Anthere (talk) 11:20, 2 November 2022 (UTC)

Adding somewhere a link to depict guidelines

In the tool. Clickable link

• Commons:Depiction guidelines Anthere (talk) 11:23, 2 November 2022 (UTC)

No Suggestions from Machine Vision

When testing the tool on isa-dev today, I didn't get any suggestions from the Machine Vision algorithm. Adding "?mv=true" at the end of the URI fixed the issue. -- Beat Estermann (talk) 15:26, 4 November 2022 (UTC)

This temporary issue has since been resolved. --Beat Estermann (talk) 08:21, 10 November 2022 (UTC)

Baseline Statistics for the ISA Tool - Request for Comments

The project team is currently reflecting on a set of indicators that could/should be tracked and reported on a permanent basis in the form of baseline statistics for the ISA Tool. Feel free to review the current proposal and add your comments directly in the Google Doc. --Beat Estermann (talk) 15:31, 22 November 2022 (UTC)

ISA Tool Workshop - Save the date: Dec 12th

Greetings.

We released a few weeks ago a new version of the ISA tool on the Test Server. The new version propose semi-automatic tagging of images, which seeks to assist community in identifying and labelling depicts statements for Commons files.

Now we are looking for feedback with regards to

• the look, feel, and usability of the tool with the machine vision / metadata-to-concept enhancements in place.
• and most importantly, the quality of the recommandations provided.

This is why we are organizing an online workshop on Monday 12th of December, 16h-17h30 UTC+1 (on zoom). We will be very happy if you find the time to join us.

More information and sign-up : https://commons.wikimedia.org/wiki/Commons:ISA_Tool/Image_to_Concept#ISA_Workshop_-_Dec_12th

Thank you in advanceǃ

Anthere (talk) 00:28, 6 December 2022 (UTC)