Japan did it.
Automatic anime images tagging with neural networks.
THE FUTURE IS NOW
http://illustration2vec.net/
http://illustration2vec.net/papers/illustration2vec-main.pdf
Impressive
Very nice
Pretty good.
Is there an offline version of this? i can finally sort my 20000+ unix-time named image collection into folders.
holy shit
bump this amazing thing
Is this the new waifu2x?
Fuck off weeb
>>51661580
waifu2x is for upscaling, this thing is for tagging.
They both use neural networks though.
>>51661587
Normies holding back technology as usual.
Neat.
>>51661591
I meant its explosiveness popularity.
>>51661062
OP how do you find this shit anyway?
>>51661580
that was about upscaling images. This is about *understanding* the content of images.
The next step is understanding actions and scenes going on in an image. pic rel.
I guess it's too obscure.
>questionable
>58.1%
Ha that's great.
>>51661507
It looks like everything is available on the website.
![[wow.jpg].png](https://i.imgur.com/pqYh0L2.png "[wow.jpg].png")
This is pretty fucking amazing.
Wasn't able to pick up the character despite being a popular one, but pretty impressive nonetheless. I hope to see an offline version.
but is it free software?
I'm impressed.
>>51661654
pretty sure it is proof of concept
>>51661642
Well the problem is that the art isn't in the style of K-ON, and K-ON is generic as fuck anyway.
>>51661654
It's free as in freedom and as in beer.
>>51661654
>The pre-trained models and the other files we have provided are licensed under the MIT License. In other words, you can freely use our models regardless of academic use or commercial use.
Holy crap
I guess it's not that good at identifying characters.
Hiro should integrate this into 4chan so we can use it to find source on images.
I'm surprised it picks up on crossdressing.
>>51661715
Integrating it into 4chanX will be easier. ccd0, where are you dude?
Nice
>>51661062
wtf... it is future.
Got the character wrong. Not surprising considering the two characters wear the same dress.
At least it can do something right.
Didn't recognize Akari.
Needs work.
>>51661779
I've had it recognize Akari a few times. it's just not very good at it.
>>51661779
That's because you didn't upload a file with a character in it.
Still didn't recognize Akari.
Can't even get her hair color right.
Got this one way off. Not that I blame it, you really need cultural understanding to get this one.
Fucking kek
>>51661821
That's an image of a water fountain and sink.
>can't tell what from
>prolly touhou
Looks like moot isn't so bad at drawing after all.
Umm... no
>>51661821
See?
There we go.
>>51661838
Well, yeah. that's more or less how it works.
>>51661849
>Kill la kill
infuriating desu
>>51661823
sides vaporized
>>51661831
oh man, you could totally train one of these to go along with that
>1girl
Wow, something wrong with this. It thinks there is a penis in this image.
>>51661838
Looks like it.
>>51661879
It's only 79.9% confident
>>51661908
Woops.
>>51661894
Got the same thing
Impressive.
>>51661924
Well can you blame it? Instances of object in mouth are 99% penises.
>>51661899
Yeah, but I'd say guessing touhou when in doubt is acceptable.
Doesn't seem to know much about older, more obscure series.
It doesn't work with /d/.
Useless.
>>51661941
True
It's good, looks like it has trouble with characters though if you give it a screencap.
>safe
>>51661949
>lucky star
My sides
>>51661941
This can't be that obscure.
>>51661916
>no humans
Sounds about right.
>>51661973
Rape is completely safe. You should try it sometime.
>>51661989
>hat
>generic features
There's no way you're not going to get touhou
>>51661743
It would be easier if there was a way to select files by URL.
>macfag
>no humans
>>51661989
It's called illustration2vec not screencap2vec. This is trained on the *boorus.
>>51661062
>anime
kill yourself
Surprised it didn't tag Maka.
>>51662026
I did this one earlier, though.
kek'd
>multiple girls
wow Im impressed
>>51661712
difficulty with characters might mostly be about difficulty with art style or alternative colors.
The happening will be a safe one
>KanColle
>>51662078
It got denpa right though.
>tags
THEY KNOW
I don't like how it can't really tell the difference between sou and Sei. Or in other words between desu and boku.
They have different heterochromia for fuck's sake.
>>51662114
Kantai collection confirmed for jewish
>>51662114
more like
>no humans
>>51662064
It's convinced that Turkeyhead is from Monogatari even with official art.
It's pretty good.
bretty good
>>51662170
wrong photo
IBM's supercomputer destroyed Ken Jennings in Jeopardy.
https://www.youtube.com/watch?v=C5Xnxjq63Zg
Glad it recognized them.
Gave a small chance that this is explicit, how the fuck does it know?
>>51662217
>smartphone users
Every fucking time
>>51662217
On an earlier image that was actually safe, it seemed to be cluing in on the slimy stuff.
Whoa
>>51662107
Fucking how?
Does it just have the full image in storage?
>>51662258
It spends most of it's time surfing /a/ and /jp/.
This is blowing my tiny human mind.
Nice
explain this to me like an idiot how does this work
Pretty good.
Looks like it can read too.
Well.
Fuck.
It's over everyone. It's all over. Machines are ready to take over. Go home. You're drunk.
Protip:
Shift-F2, then type 'screenshot --fullpage'
>>51662385
So far it seems to recognize Mai-HiME, but not the characters.
>>51662337
get your master's degree in computer science
then you'll understand how it works
's not that good
>>51662467
It's for weebs.
>>51662337
https://en.wikipedia.org/wiki/Artificial_neural_network
https://en.wikipedia.org/wiki/Convolutional_neural_network
>>51662483
FUCK YOU 100%
Really does seem to need colour.
At least it got the safety of this right.
Wow what a disaster
Works well
I do this kind of work as part of my thesis in my university. I'm currently making a prototype with other students that restores images through machine learning.
All these types of creations are bound to be imperfect. The interesting thing about machine learning is that it becomes frighteningly accurate in solving problems, but will never reach anything reliable, let alone perfection. It's also limited by the training data you use and it's behavior is unpredictable outside those bounds.
It's amazing how well this technology works, but personally I don't think it should be applied to anything that can't have a large margin for error. Kind of what we're seeing here. Lot of really good catches, but a few aren't great even if the image looks easy to tag.
Excuse my autism...
>>51662537
How long until neural nets recognise faces?
Okay this is a stupid question since this is how facebook's face recognition software works.
Does it spook anyone else that facebook can take a picture of almost anyone in the world and put a name to it?
>>51662566
>How long until neural nets recognise faces?
Ten years ago.
>>51662605
Yeah but I mean everyone's face. Actually having the ability to tie a name to it rather than just recognising that they are of the same person
Well I will admit the difference is subtle..
>>51662618
>Actually having the ability to tie a name to it rather than just recognising that they are of the same person
... the name was added by someone. Like the account owner. Which is absolutely trivial database stuff *after* you did the image recognition and grouped up images.
It doesn't derive it by computing the universe.
>>51661823
Holy shit.
>>51662618
You can't figure out someone's name by looking at their face, what the fuck?
nice
>dutch angle
ARE YOU FUCKING SERIOUS?
HOW
>>51662618
>>51662566
Like the other anon said the technology is very much there and more. Someone in my university did work recognizing facial expressions.
The problem with putting names to faces is as much about improving tech we have as it is about ethics. Like I said machine learning needs a lot of training data to be functional and that means pictures of people and their personal info. I think facebook just uses tagging right now, not sure. What's scary about facebook is the massive amounts of metadata they hold. Not that I'm gonna put on my tin-foil hat anytime soon, but it could be abused by someone who isn't facebook.
THANK YOU BASED JAPAN
The future looks bright.
>>51662940
It can also be abused by facebook, if you ask me.
Obviously we need legal constraints on everything. Face recognition, dick recognition, tracking of physical location of mobile phones and cars, strangers and acquaintances we talk to -even on the internet- ... it all should actually not be monitored and catalogued.
>2girls
W
H
A
T
>>51663298
>>51663308
It's saying that mako is in there twice, hence the 1girl 2girl
>>51662938
koek
>Otoko no ko
How'd it even get that?
I guess 'draw a girl and call it a boy' isn't a valid criticism anymore.
>>51664062
It senses your gayfaggotry
How the FUCK does it detect bikinis
i suspect it looks up the images in different sources (boorus etc) but first, it looks if the filename is familiar with any of the sources
so if you look up this image through neural networks, it'll first look up the filename "0d4f98281cc4774ce66417dc707ee4bc" which exists for danbooru and it'll grab the tags from there. however, if it can't look up the filename because it doesn't exist, it'll search for the picture via IQDB or something
>>51664090
gayfaggotry is the best faggotry.
>touhou
>bunny ears
wut
This thing has a problem with arm placements.
>>51664136
No it doesn't it literally decides what tags it gets, not from some site.
I used a picture that is not on a single site in the world, still gave it all the proper tags it should get, magically.
>>51664164
what sorcery is this?
I've tried at least four or five lain images and it keeps insisting that she's a boy
Damn it
this needs to be a database so i can search with tags for all of my pics
>>51664136
This isn't how it works at all.
Try it out dude.
My recommendation is to draw a character yourself. Just draw one and see if it recognises it. Even if you can't draw for shit, it'll prove that it's a neural network.
Anyway, the people who made this give you the inner workings of this. So they'd have produced a working neural net, published exactly how they did it and it working, and then just decide to write a booru filename finder.
Yeah nice theory idiot.
>>51664136
that's the way it was trained, but not the way it operates now.
also the filename is the md5sum of the file, it wouldn't need the name for anything
Took a few tries before it recognised a character but impressive nonetheless.
>inb4 copyright holders start using roaming bots to scan anime images and send DMCAs to everyone who posts images of their shit
>>51664267
inb4 ttp
How was the model trained?
This is the most important thing.
If they manually specified where every tag is on the picture (ie for long hair, a person would manually draw a rectangle over character's hair on the picture), it's not impressive.
If they simply fed pictures and a list of tags for each picture to the model, then it's amazing.
>>51664308
>If they simply fed pictures and a list of tags for each picture to the model
This is how neural networks are trained and this is what makes them important to computing.
So yes, it really was trained just by showing it pictures and tags.
>>51664308
> witch hat
HAHAHAHAHAHAHAHAHAHAHA
STUPID!
>>51664303
TPP + this = imageboards are kill
>>51664303
There are less than 2 months to go to fill up your storage drives
>>51664326
Google's thing that created dogs' nightmares was trained on specifically pictures with just one feature on each.
>>51664332
You know traffic cones are called witch's hats colloquially in England and Australia right?
>>51664334
tpp + anything = everything is kill
reminder that if you're not using legacy captcha you're training the botnet
Pretty impressive tb.h fa.m
>>51664355
No, I did not know that.
>multiple boys
guess im a gay
>>51664433
>JOHNATHAN JOESTAR
major kek anon thanks, it's probably the poses
>>51664433
>all this jojo faggotry ruining muh dbz
>>51664431
Well it got the eyes
It guessed everything.
Without knowing the training set, though, it maybe be very easy to overestimate how good this thing is. Maybe it's trained on all booru picture, and that's why we're getting such good results.
The best way to test this would would be to use pictures that were drawn after the model was trained.
>>51664433
>Johnathan
>Jospeh
>Jotarou
>giovanna
>Jousuke
>FUCKING DIO
Im fucking dying, holy shit
someone should create a software for this
WE ALWAYS NEEDED THIS
It's detecting everything I upload as Kancolle or Touhou, despite not being either Kancolle or Touhou.
1). 0f58281d934a56ec410209ff9bdda990.jpg
ERROR: Bad Gateway
looks like its a scam
>>51664433
more jojo
>>51661062
Didn't read. Does this mean that now I'll be able to hide weeb posts automatically? Would be awesome.
>>51664561
If you could integrate this into a 4chan extension, yes.
wew lad
Complete shit
>>51664497
>Maybe it's trained on all booru picture
It's not. A bunch of stuff on the booru wasn't identified well.
>>51661625
Shae. I've always wanted to know what this one is.
>>51664707
>that feel when
>>51661614
Would be neat for a "what the hell is going on" device for blind people. Maybe in 15 years.
>>51664640
>draw girl
>say it's a boy
k
>>51661808
Ah! I've been looking for that image forever! Please, please upload it somewhere!
>>51662078
low confidence. so it's not wrong
This one surprised me a bit
>>51661943
which one is this again? i thought it came out a couple years ago, if that
>>51662319
alice is a common theme in nip art, so alice cosplay is easy to associate for the NN
How can it recognize cirno from that angle?
>>51664747
It's Happiness! It came out in like 2008
No, wait, it came out in 2006. That shit is old.
>>51661949
Wtf, is that sperm? Eels? wtf is that?
>>51661949
there isn't much furryshit on the boorus. you would have to train it on a different image set
>>51664723
Exactly.
>>51664704
Sure you did. The only time it would've been posted was long ago and you would know what it was from at the time.
>>51664803
Not really, I usually lurk. And perhaps it wasn't specifically on this site, FYI. I've seen it before and usually when I ask in a thread (granted it was a while back), I get newfagg'd.
Oh well. I'll see what Google/TinEye pulls up if you're not going to tell me. Thanks anyway though, anon.
Wild differences in prediction quality leads me to believe it only gives good results for images from the training set.
>>51661821
>>51661779
Literally who?
>>51664842
Seems more like it just can't do generic animu very well. Or screencaps, or official art.
>>51664874
It's a meme. There's no one named Akari in those images.
>girls
heh
>>51664884
>>51664874
It also detects Tsukihime anime for me.
>>51664497
There is a list in the repo of all the tags it has.. and the coffeemodels are available...
https://github.com/rezoo/illustration2vec
>>51664919
Well, what matters is what images were used to build the model. Because if the model guesses tags correctly for images that were in the training set, that's not very impressive.
>>51664931
While true, I've fed so much obscure stuff into this that it's impossible for it to all be from the training set.
>>51664931
>>51664497
>>51664842
>http://www.pixiv.net/member_illust.php?mode=medium&illust_id=53866716
>12/3/2015 23:00
>uploaded to pixiv today
science, it works bitches
>>51664954
>>51664959
I guess you're right. The effect is so amazing that my whole existence is screaming at me that I'm being scammed.
>>51664803
But anon I want to know
>>51664267
>what is fair use
>>51662127
They look much alike. I wonder which one is Sou and which one is Sui in this picture according to the algorithm.
>>51661949
WHAT IS COMING OUT OF HER TITS
>>51664971
Two pixiv images, one with cum and one without.
>>51664971
it's not that amazing. your brain has higher accuracy, needs fewer watts and does unsupervised learning instead of needing a painstakingly tagged, huge training set.
>>51665002
My bad:
>>51665002
I think you fucked that one up
>>51665007
I can't spawn a thousand copies of my brain to do stuff for me. Unfortunately. It is amazing.
>>51665007
Also,
>unsupervised learning
Years and years of supervised learning.
>>51665034
that's more a property of software running in silicon instead of wetware in general. not specific to NNs.
>>51665046
your domain-specific knowledge of mongolian stone carvings has been acquired through unsupervised learning.
>>51665067
It is amazing because it lets me do things previously impossible. Anon. Why are you even arguing that it's not amazing?
>>51665077
because it has a fairly high error rate and only identifies things that are extremely popular anyway.
the more niche the source material is the less useful it gets. it's exactly the opposite of what you need.
in other words, it can only tell you what you probably know already anyway.
>>51661690
>1 girl
>>51661777
Trips
>>51665104
Interesting.
Very slight differences.
What's interesting here is that the slight change in the thigh highs causes massive chances in the detection. It really detects the thighhighs.
>>51665104
You'll be seeing a LOT of that.
>>51665101
Well it's not so much for identifying but for tagging content automatically. Identifying is really hard.
>>51665101
Ah, you're talking about character detection. But it guesses a lot of tags correctly, which, for example, can be used to search a huge set of images without having to tag them.
Not bad except >1girl
>>51661606
>2 girls
>>51665101
well to identify niche things it needs to learn how to spot them. With a bigger learning database it could identify the niche characters.
But I mean this is a combinatorial problem almost- there are just so goddamn many characters and shows.
I'm really not sure how it got this one
Impressive.
>>51665116
even for automatic tagging you would have to discard all but the highest-confidence results. which are fairly generic. all the niche interests would still have to be tagged manually.
>>51665122
i've fed it some images from my fap folder. it detects none of the fetishy aspects that are of interest to me, probably because they're not tagged 10k+ times.
And the stuff it does detect often is somwhat off. e.g. it detected a cupless bra/lingerie top as apron.
So if I were looking for fap material it would miss a lot of stuff and also emit a lot noise.
It's not amazing. It's moderately useful at best.
>>51665163
Idolshit is popular so it probably had a lot training on it. It seems to do good with 2hu and LL as well.
>>51665144
the point is that humans can identify learn to identify things based on a handful of samples. based on a single character sheet really. the NN needs hundreds or thousands of samples to even get a barely tolerable detection rate.
>>51665189
Personashit too?
>70% legend of zelda
it needs some work
>>51665183
I just think it hasn't been trained on much NSFW stuff.
With a biggest training base I think this thing would become awesome.
Maybe something like this but specifically trained for porn would be great. IDing semen demons and such.
>>51665198
I have no idea how popular it is but if it is, I would guess so. 2hu, LL, and Idolshit seem to pop up often when it's guessing.
I think it's pretty decent.
A great starting point.
>2girls
>yuri
>kiss
What a goof
>>51665040
for_window [class="Firefox"] border none
or
bindsym $mod+x border none
bindsym $mod+z border normal
And hide_edge_borders both
is nice for when you only want the title bar to show for windows.
>>51665211
>I just think it hasn't been trained on much NSFW stuff.
I'm pretty sure they fed it the entire database of gelbooru, so it should be decent enough at NSFW.
you kind of forget that human tagging on boorus is far from perfect. some images are well tagged, others not at all.
>>51665322
Gelbooru isn't particularly diverse. I mean there are like 100 gatchaman images and 500,000 2hu images.
The diversity in nsfw is likely to be similar.
>>51665136
>draw character 100% like a female
>expects it to know that it's a trap
not even a human could guess half of those.
>>51665362
What's interesting is that if it gets the character right it's more likely to guess that it's a boy.
>>51665211
I dropped in some stuff from my high detail art folder (all from pixiv). it only found the most generic things.
>alice in wonderland themed
>only finds eye color, dress, 1girl etc.
>halloween-themed image full of pumpkins
>thinks its pokemon
>book shop with 1 girl manning the cash register
>it's not even confident about the girl, spots nothing else of relevance
It's interesting technology, but it's not good enough and the amount of training data it needs to spot something is a real problem for the long tail of tags.
And you can't simply throw more data at it because if it runs on GPUs you are limited by the gfx ram. Once it's full you can't make a larger network anymore and thus you're limited how many hidden variables this thing can learn.
>>51665374
Then perhaps more diverse data? It seems to get Kancolle REALLY easily.
Just needs the clothes.
>>51661671
>large breasts
HOW DO YOU EVEN KNOW
>>51665406
>>51661062
bunch of faggots. and don't let anyone tell you otherwise
