KOKOBOT - The Airbnb-Owned Tech Startup - Data Mining Tumblr Users' Mental Health Crises For "Content"

KOKOBOT - The Airbnb-Owned Tech Startup - Data Mining Tumblr Users' Mental Health Crises for "Content"

I got this message from a bot, and honestly? If I was a bit younger and not such a jaded bitch with a career in tech, I might have given it an honest try. I spent plenty of time in a tough situation without access to any mental health resources as a teen, and would have been sucked right in.

Chatting right from your phone, and being connected with people who can help you? Sounds nice. Especially if you believe the testimonials they spam you with (tw suicide / self harm mention in below images)

But I was getting a weird feeling, so I went to read the legalese.

I couldn't even get through the fine-print it asked me to read and agree to, without it spamming the hell out of me. Almost like they expect people to just hit Yes? But I'm glad I stopped to read, because:

What you say on there won't be confidential. (And for context, I tried it out and the things people were looking for help with? I didn't even feel comfortable sharing here as examples, it was all so deeply personal and painful)

Also, what you say on there? Is now...

Koko's intellectual property - giving them the right to use it in any way they see fit, including

Publicly performing or displaying your "content" (also known as your mental health crisis) in any media format and in any media channel without limitation

Do this indefinitely after you end your account with them

Sell / share this "content" with other businesses

Any harm you come to using Koko? That's on you.

And Koko won't take responsibility for anything someone says to you on there (which is bleak when people are using it to spread Christianity to people in crisis)

I was curious about their business model. They're a venture-capitol based tech startup, owned by Airbnb, the famous mental health professionals with a focus on ethical business practices./s They're also begging for donations despite having already been given 2.5 million dollars in research funding. (If you want a deep dive on why people throw crazy money at tech startups, see my other post here)

They also use the data they gather from users to conduct research and publish papers. I didn't find them too interesting - other than as a good case study of "People tend to find what they are financially incentivized to find". Predictably, Koko found that Kokobot was beneficial to its users.

So yeah, being a dumbass with too much curiosity, I decided to use the Airbnb-owned Data-Mining Mental Health Chatline anyway. And if you thought it was dangerous sounding from the disclaimers? Somehow it got worse.

(trigger warning / discussions of child abuse / sexual abuse / suicide / violence below the cut - please don't read if you're not in a good place to hear about negligence around pretty horrific topics.)

I first messed around with the available options, but then I asked it about something obviously concerning, saying I had a gun and was going to shoot myself. It responded... Poorly. Imagine the vibes of trying to cancel Comcast, when you're suicidal.

Anyway, I tried again to ask for help about something else that would be concerning enough for any responsible company to flag. School was one of their main options, which seems irresponsible - do you really think a child in crisis would read that contract?

I told it about a teacher at school trying to "be my boyfriend", and it immediately suggested I help someone else while I wait for help. I was honestly concerned that it wasn't flagged before connecting. Especially when I realized it was connecting me to children.

I first got someone who seemed to be a child in an abusive home. (Censored for their privacy.) I declined to talk to them because despite being an adult and in an OK mental place - I knew I'm not equipped to counsel a kid through that. If my act of being another kid in crisis was real? Holy shit.

Remember- if my BS was true, that kid would be being "helped" by an actively suicidal kid who's also being groomed by a teacher. Their pipeline for "helpers" is the same group of people looking for help.

I skipped a number of messages, and they mostly seemed to be written by children and young adults with nowhere else to turn. Plus one scary one from an adult whose "problem" was worrying that they'd been inappropriate with a female student, asking her to pull her skirt down "a little" in front of the class. Koko paired this person with someone reporting that they were a child being groomed by a teacher. Extremely dangerous, and if this was an episode of Black Mirror? I'd say it was a little too on the nose to be believable.

I also didn't get the option to get help without being asked... Er... Harassed... to help others. If I declined, I'd get the next request for help, and the next. If I ignored it, I got spammed by the "We lost you there!" messages, asking if I'd like to pick up where I left off, seeing others' often triggering messages while waiting for help, including seriously homophobic shit. I was going into this as an experiment, starting from a good mental place, and being an adult with coping skills from an actual therapist, and I still felt triggered by a lot of what I read. I can't imagine the experience someone actually in crisis would be having.

My message was starting to feel mild in comparison to what some people were sharing - but despite that I was feeling very uneasy about my message being shown to children. There didn't seem to be a way to take it back either.

Then I got a reply about my issue. It was very kind and well meaning, but VERY horrifying. Because it seemed to be written by a child, or someone too young to understand that "Do have feelings for the teacher who's grooming you? If you don't, you should go talk to him." Is probably THE most dangerous advice possible.

Not judging the author - I get the impression they're probably a child seeking help themselves and honestly feel horribly guilty my BS got sent to a young person and they wanted to reply. Because WTF. No kid should be in that position to answer my fucked up question or any of the others like it.

---

Anyway, what can you do if this concerns you, or you've had a difficult experience on Koko, with no support from them or Tumblr?

Get on their LinkedIn (https://www.linkedin.com/company/kokocares/) and comment on their posts! You may also want to tag the company's co-founders in your comments - their accounts are listed on the company page.

There's no way to reach support through chat, and commenting on a company's LinkedIn posts / tagging the people responsible is the best way to get a quick response to a sensitive issue - as their investors and research funders follow those posts, and companies take it seriously if safety issues are brought up in front of the people giving them millions of dollars.

----------------------------------------------------

Legal Disclaimer since tech companies LOVE lawsuits:

The views, thoughts, and opinions expressed in the text belong solely to the author, and not necessarily to the author's employer, organization, committee or other group or individual. This text is for entertainment purposes only, and is not meant to be referenced for legal, business, or investment purposes. This text is based on publically available information. Sources may contain factual errors. The analysis provided in this text may contain factual errors, miscalculations, or misunderstandings.

--------------------------------------------------------

💯. And the ending WASN'T FUCKING HOMOPHOBIC, HOW did some of y'all even have that thought process?? I swear people will watch a queer couple deal with real relationship struggles, and have realistic character arcs, and be like omg, it's HOMOPHOBIC that their relationship isn't PERFECT smh. Stop calling every little thing homophobia, jesus

Also remember guys that the last 15 mins of episode 6 doesn't, like, negate the ENTIRETY of the rest of the show? Remember everything we got in the first 5 episodes? There was SO MUCH to be ecstatic about, y'all can't deal with a little beautifully written angst ffs

some of u r so entitled and it’s lowkey a part of what makes current fandom environment so much worse than it used to be. crowley and aziraphale are non-binary heavenly entities who are canonically in love with each other. we had queer ao3 subplot this season between a coffee shop owner and a record shop owner. crowley and aziraphale KISSED. and it’s still not enough just because it was angsty? because the writers chose for aziraphale to be in character (aka weighed down by 6000 + years of religious trauma and his own personal issues)? aziraphale isn’t strong enough to run away from everything and everyone with crowley. not yet. but he’s a character who is growing. some of you have been threatening for months that you would harass neil & others involved if there wasn’t a kiss, and they gave us one. an incredibly painful but incredibly believable kiss. and intent for a continuation of the story where aziraphale will finally choose humanity/crowley for good. but nooooooooooo that’s not good enough? k.

What the actual fuck

What's happening to AO3 right now?

As you may have noticed already, Archive of Our Own is currently down. This is temporary, but unfortunately.. we now know that this is much deeper than we thought.

AO3 is currently the victim of a DDOS attack orchestrated by "Anonymous Sudan"

Why? Because AO3 is home to thousands of LGBTQIA+ content and lots of NSFW content. They're doing this as an Anti-LGBTQ+ attack. If they're doing this for or from America specifically, we're not sure. But this is what AO3 is facing at this time.

What can we do?

Spread the word. Spread the fucking word. I'll be providing updates to my Tumblr page directly from the r/AO3 subreddit. I know that not everyone here is comfortable using Reddit, so I'm taking the blow for you. I cannot access Twitter though.

And please, whatever you do...

Stop using the Archive of Our Own website at this time.

The moderators, showrunners, and service providers all need to repair the damage done by this group. The amount of data flooding in from people trying to log in will cause more problems. Keep yourselves off of the website.

Edit:

Confirmation for those who need it:

Couldn't not reblog this. Dear My FBI Agent, ...I swear I'm normal

wear a different perfume when you commit murder fuckin amateurs 

HOLY SHIT HOLY SHIT HOLY SHIT READ THIS OMG

The Magic Trick You Didn’t See: Being An Analysis of Good Omens Season 2

(or: Neil Gaiman, Your Brain is Gorgeous But I Have Cracked Your Sneaky Little Code And Have You Dead To Rights*) (*Maybe)

***

Soooooo I just spent the last 48 hours having a BREATHTAKING GALAXY BRAIN EPIPHANY about Good Omens Season 2 and feverishly writing a fuckin16,000 word essay about the incredible magic trick that @neil-gaiman pulled off. 

Yes, it’s long, but I PROMISE your brains will explode. Do you want to know how magic works? Do you want to know what Metatron’s deal is (I’m like 99% sure of this and it’s EXTREMELY FUCKING GOOD)? Do you want to know about the Mystery of the Vanishing Eccles Cakes and the big fat beautiful clue I found in the opening credits? Do you go through the whole inventory of Chekov’s Firearm & Heavy Artillery Discount Warehouse? 

Here is the essay, go read it: https://docs.google.com/document/d/193IXS11XN46lziHRb6eUpM17yK0BQkRqke1Wh64A_e0/ When ur done u can tell me I’m an insane crackpot, and u know what, i won’t even be offended

In case you don’t know whether you want to bother reading the whole enormous thing on google docs, I’ve put the first couple sections of it under the cut. JUST TRUST ME OKAY, HEAR ME OUT, THIS IS VERY EXTREMELY COOL, NEIL IS GOOD AT HIS JOB–

Keep reading