Bots flooding GC
 

All the downtime today (August 26) was due to bots, specifically AI data scraper bots as far as I can tell but might also be search engine related.

They were sending far too many bots way too fast and it was making GC unusable. Basically caused an unintentional DoS (denial of service) "attack" on GC. So I turned GC off while getting that all sorted out.

The IP addresses were from a major Internet company (hundreds or maybe even thousands of IPs) and were being used by a different major Internet company.

TikTok / Bytedance apparently must be really quite interested in the conversations going on here at GreekChat!

Must be due to #bamarush!

Can they be blocked?

Is that what happens? The more interesting the site, the more bots that show? I don’t understand what they collect and the importance of what they collect. And then where does what they collect go?

Usually, yes. But not always. Depends on how determined the bot/spider operators are.

I rerouted these particular bots in a way which should minimize their impact on GC. Similar to what I did with the bots from approx 2 weeks ago.

Yes, that's pretty much how it goes as far as I know. At least with search engine type bots as search engines want to direct their users to interesting & useful sites.

GoogleBot is a good example of a beneficial bot as that bot indexes sites & helps increase listings in search which might result in more new website visitors.

The bots from Bytedance / TikTok seem to be AI data scraper bots. But might also be search engine related as well.

They might be training AI / artificial intelligence chatbots using conversations that they scrape from forums all throughout the Internet.

Around a decade ago the world entered a new era of AI. Way back in the day one way AI was made was with programmers creating decision trees to determine what to do depending on different scenarios and that would be many levels deep. Modern AI doesn't do that. Instead, modern AI gives the engine, the neural network, as much labeled data as possible which the AI trains on in order to recognize patterns within the data. The AI is trained & tested over and over, repeatedly. Constantly refining, tweaking the neural net while continually improving accuracy. AI recognizing patterns in images is a good example of this type of training.

With GC being text based conversations, AI data scraping on that is likely to be for training AI language models. Basically very sophisticated chatbots. Similar to ChatGPT if you're familiar with that. Huge amounts of data, as much as possible, is used in training these AIs.

My guess is that if used for training AI, the data is refined and is used for training AI now and maybe even in the future. Could be one giant combined dataset of many different online forum discussions that they continually add to & improve. All that data may be used for training AI, but might never actually be seen by anyone other than the AI researchers and/or data engineers working on it.

Based on your post, John, it sounds like bots will be the reason for a future AI takeover. I mean, with the information they’re getting, it’s eventually going to be used for AI in everything. That’s the way it reads.

I'd probably blame the future AI takeover more on math & programming in general. Modern AI probably boils down to highly accurate pattern matching that is enabled by complex math & statistics. Programming enables it to be automated and fast.

But it does need lots of data for training, so bots are a part of it.

Modern AI is such a revolutionary advancement that any businesses (or governments) that can benefit from it need to adopt it or eventually they will be surpassed. The disruption will be similar, probably more significant, to when the Internet started being commercialized and ecommerce was disrupting businesses in all different industries.

Yep. I totally get that AI is going to be big business, if it isn’t already. It’s just that the possibilities for future AI systems are limited by what we currently know about the nature of intelligence. And then, I think it’s going to be a huge mess, because based on the way I see things happening with AI, it seems like all evidence suggests that human and machine intelligence are radically different. Know what I mean? Don’t you think it will be more problematic than beneficial, based on that? I mean, you’d know better than me, but I just don’t see the benefits.

I think "AI" might be more correctly referred to as "APM" / "artificial pattern matching." AI systems aren't "intelligent" but they can in many ways now replicate the patterns of intelligence well enough to convince us of AI "intelligence" and that ability/feature of AI will certainly improve in the future. Future AI will be so good at doing that most people will likely believe that what they are seeing is actual intelligence when it's just pattern matching.

In regards to how the brain does what it does and how AI does what it does I would probably agree that it's radically different. But the end result I think is quite similar. Humans, as far as I understand, do much of our learning based on patterns that we perceive with new or more complicated learning often or usually based on the patterns that we previously perceived. That is very similar to AI in that AI is an extremely sophisticated pattern matching calculator, in a sense.

Great question. Probably both more problematic and more beneficial. But I'm not sure which one will be more significant.

I guess in comparing possible end results... AI could make life significantly better, more productive, more advancement, more efficient, more possibilities, etc. But future AI could also result in disaster, or maybe be used by people in order to cause a disaster scenario. So from that perspective, maybe AI will be more problematic because of the potential for serious harm.

But that cat is out of the bag, no going back in. Even if it's regulated it's probably too late. Those who are regulated from developing new AI will simply be those who lose out to the people who take advantage of the technology.

Consider Google's AI AlphaGo beating world master champions in the game of Go. AlphaGo was trained based on actual games between people. Then Google AI devs went and made AlphaGo Zero which was given the rules of Go and trained itself, basically played itself for a period of time and learned to master the game that way. AlphaGo Zero defeated AlphaGo significantly. I think at first it was just by a relatively small margin, then after some changes AlphaGo Zero won 100% against AlphaGo. I don't know the exact history of it, but it was something like that.

So that was a game that an AI mastered versus people and another AI completely defeated the first AI.

Consider that the military has war "games" for training & analyzing scenarios, etc. The US military not only should be using AI, they must be. They have to. Because if they are not and an enemy or future potential enemy develops AI that can defeat all the human military war game "players," like the AIs in the actual game Go, the side that does not have that advantage probably loses. And it might be that the government which reaches AI superiority first is the one that wins it all.

Then there are governments that will use massive data collection & AI training/development against their own people.

Lots of great benefits but also lots of very serious potential problems.

^^^ Oh wow! I didn’t look at it from any of the perspectives you’re looking at it from, John. So true, and you make a lot of sense. I think greed will be the downfall of it, though. I mean, for like the other side of the benefits. The bad side. It’s just that a lot of companies now launch AI teams because they’re afraid of falling behind other companies/competitors, without fully knowing where or for what purpose they’ll use AI. And then too, a lot of companies pretend to use AI when they don’t, just to increase their chances of obtaining funding. That’s the greed part. That’s the part I think will get worse. There’s like also a fair amount of general confusion about what AI can and can’t do.

What’s interesting though is that we now use it a lot, daily, sometimes without even realizing it. Do you think it can or will get out of control? I mean, right now AI is completely under human control, but in the future, it might not be under our control anymore. Seems like eventually every single task is going to be done by AI.

Definitely yes to the first question and probably yes to the second.

In some sense, there have been AIs which temporarily got out of control until they were reigned in. That would be some computer viruses. Although they aren't modern AI.

Eventually there will be modern AI based computer viruses, though.

With that in mind, there will be AI trained to hack computers. I'd be really surprised if that hasn't already been accomplished. If it hasn't or at the very least if it isn't being worked on & developed then many people in the computer security side of government (and cyber warfare) need to lose their jobs. Hacking is like solving puzzles... puzzles are games, basically & AI has already mastered many games, so why not hacking. Problem is that if we don't do this what if an enemy does. Train the AI to hack, train the AI to find new vulnerabilities, train the AI to infect, to evade detection, stuff like that and we have a real problem, especially if integrated into a virus.

Then there will be AI trained to protect against AI that is trained to hack. And who knows which side will win that battle.

But it doesn't even need to be an AI designed to do bad things for it to get out of control. Could be a completely well intentioned AI but some aspect of the logic was missed or not protected and the AI determines a solution to whatever problem it is solving is to do something really terrible.

It all seems like science fiction but we probably aren't too far off from having self driving cars that are safer than people driving. AI controlling things that have the potential to do a lot of damage becomes more and more real over time.

This is interesting, John. I remember when I was a flight attendant, our airplanes had auto-land, so they could land themselves. I mean, the pilots still had to make a few adjustments, but not many. Human error is what generally causes plane crashes, when they rarely happen. I’m sure the same would be true with self-driving cars and trucks.

I hope a massive cyber attack where it affects how we live never happens. There are so many things we take for granted that can be wiped out by cyber attack from an enemy.

My guess is that auto-land is more like an advanced type of cruise control. Although maybe those types of systems have been integrating moderin AI in recent years.

Self driving cars are based on modern AI tech which is quite different in how it functions.

I don't know if self driving cars are yet statistically safer than people driving. But over time I'm fairly sure that self driving AI will approach then eventually surpass the ability of people to drive safely.

There have been many, but for the most part were able to be resolved, as far as I know.

One example is the Colonial Pipeline crypto ransomware attack in 2021. That cyber attack had an almost immediate significant impact, but the company was able to resolve fairly quickly. When that pipeline was shut down it impacted most of the US East Coast and several States declared States of Emergency due to it. The day after that cyber attack the company paid $5 million in ransom and luckily the hackers provided the fix after getting their ransom.

There have been many companies and different levels of government that have had to pay ransom to hackers to recover essential systems. I recall reading of a number of companies that went bankrupt after not being able to recover from cyber attacks.

That's for sure.

I remember that. Ok, so I read this post a few days ago and I began to wonder since it’s like nearly impossible to catch these criminals who do this, why don’t they just keep doing it and requiring ransom money? It’s not like they can get caught. I remember when that happened and I wondered the same thing when it happened.

Around a year ago I watched an interview of a guy who was part of some FBI task force for investigating high profile / most wanted hackers, or something like that. He said that we will never catch them and the only ones who do get caught are usually because they made some sort of tiny mistake. There's a documentary about the silk road hacker who was basically caught this way, made a little mistake.

I read something about this a while back. Seems to be that if the hackers get a reputation for not honoring their ransoms then people / companies / governments will probably stop paying future ransoms.

I’m going to look for that documentary. I’d like to really see that.
I meant if they continue to do it the same way they’re doing it now. So, honor their ransoms by fixing what they screwed up, after they get ransom money. What I’m saying is why not keep doing what they’re doing repeatedly, keep honoring the ransom, then do it again. It seems like they just get the ransom and stop. Why not keep doing it since it’s very difficult to get caught.

They continue upping the ante. In the last briefing I attended with someone from the NSA, they were talking about how it has changed. Originally, ransomware just encrypted all the data and then they had you pay to get the decryption key. But people started doing better backups and could just restore their systems so they stopped doing it that way.

Then they began extortion scams- so they'd say to pay the ransom to get your data back AND to prevent them from exposing the data they had on the dark web.

It's difficult to get indicted/convicted because most of this is happening from outside the US. And a lot of it is nation state.

NOW they have gone a step further and offering protection services from other scammers if a company will pay them monthly- very much like organized crime rings operate. We'll give you your data back, we'll keep it off the dark web and we'll give you protection.

This is why I have job security...

So, then this happens all the time? Seems like it. That would make me so mad. They seem like they have an easy job and an easy way of stealing money, literally without getting caught. I wonder why they haven’t gone after any banks, unless they have and it just didn’t reach the news.

They go after the banks in different ways. Distributed Denial of Service attacks were rampant in 2019 and 2020, trying to disrupt online banking. A banking security officer was saying that the chief information security offers from the banks/financial industry have a call every morning to discuss attacks they are seeing. They collaborate really well and warn each other so they can better defend it. It's constant though. Job security for people in cyber security- no shortage of work for us, truly.

There may be few documentaries about how they caught the silk road guy. I don't recall if it was something I saw on Netflix or YouTube, probably YouTube. When they finally closed in on him it was in a library, he was surrounded by maybe a half a dozen undercover people. There was suspicion that he had setup a way to immediately lockdown his computer with a key combo or something similar, so they couldn't just approach & arrest him as they needed the evidence on his laptop. They ended having 2 agents posing as a couple sitting nearby, the lady slapped the guy and it got the silk road guy's attention long enough for the other agents to step in a secure his computer / prevent him from hitting the key to lock it down.

There's also an interesting interview by one of the agents involved here: https://www.youtube.com/watch?v=4KiO8GRgwDk I haven't watched that entire interview but there are clips floating around. Some parts I did watch included where he talked about only the hackers who make mistakes get caught and/or something about catching the low hanging fruit, but the others have not been caught. Something like that.

There's another hacker story where the guy was caught & in jail, working on his own case. Discovered that the gov used some device to masquerade as a cell phone tower, they drove around something like a 2 square mile area collecting call data from a huge number of people. I think it may have been 100,000+ or so. I may be a little off on these facts as it's been a while since I learned about this stuff. That hacker I believe got off because the gov did that mass data collection, trying to find the hacker, without a warrant or something like that. The hacker or maybe his lawyer just happened to stumble upon the name of the device out of thousands of pages of documents.

Oh, they do keep doing it, just to other people / other companies, etc. I imagine that there may have even been victims who were hacked like that repeatedly, by different hacker groups.

That's really interesting and makes sense. I often wondered how the ransomware groups would strike back if people/companies/governments ever figured out how to secure their ability to recover when they are hit instead of paying the ransom. So now if they can recover or not they are extorted that way.

I recall learning about some major hacks where the hacker group spent months working on it. No idea how many people may have been involved in each hack or how much total time is put into it, but they do often walk away with millions. My guess is that many of these cases may also involve insiders as well. If a hacker group is going to get millions I imagine they might bribe insiders with huge payments for help gaining some kind of access & pulling off the hacks. Some of those "accidental" clicks on phishing links may not be so accidental.

Way back in the day, approx 2005ish, there was a Wired Magazine article about the larger botnets at the time. They were being used I think against online casinos & banks. The victims were just paying up because the ransome was less money than what the online casinos were losing each day while being offline. I couldn't find that specific article but if anyone is interested just Google "site:wired.com ddos botnet" and a bunch of related articles will be in the search results.

⬆️ John, this is really crazy. The video is long but still worth watching. I wonder if there’s ever a single hacker. I mean, someone that doesn’t work with a group of hackers. If I was a thief, I’d just do it once and live off of the first millions of dollars, and be done, lol.

But seriously, if I was that computer literate, I would just find a really high paying tech job. I don’t know why they just won’t do that, instead of risking their freedom. Being that intelligent, you could really negotiate what you want.

Have you ever seen the move “The Core”? There was a hacker in the movie who got caught, but then the government forced him to work for them or go to prison. It was kind of a cheesy movie, but some parts were ok.

That’s gotta be really stressful, though.

My guess is yes, there are probably many lone wolf type hackers. Mainly because other people being involved probably opens up way too many loose ends. Or, at the very least, when they work with others they still remain anonymous to those they work with. The silk road guy was very close to being a lone wolf, even the people who were working for him didn't know anything about him, as far as I recall.

I wouldn't be surprised if there are some who have done it that way. Pull off one big heist, then drop off the radar forever. For some reason this is making me think of the end result of the hack in the movie Office Space.

Yeah, I remember the hacker from The Core. Will never forget that guy after he ate the french toast in Road Trip.

In that YouTube interview the agent talks about catching one of the silk road guy's accomplices. I think they made that hacker work for them for nearly a year or so.

Somewhat related but a massive spam bot actually took over Greek rank, specifically the SMU page. It’s been nuts to see hundreds of pages of the same spam over and over again. I assume it has to be using a program to spam that hard.

We had one about 10 years ago that was Indian astrologer spam. It was awful.

And here I was thinking that when they’re working as a group, they all know about each other. Like, they’re in a hidden room together doing this. I’m so naive, lol.

lol
He was surviving off of hot pockets in the movie “The Core”.

I watched about 45 minutes of that video. I’m going to continue watching it this weekend.

There's a mix but vast majority is programmatic. Years ago I read of spam operations that hired people in extremely low wage countries to post spam and other operations that used these low wage employees to solve the CAPTCHA anti-bot puzzles all day and the bots did the rest.

There are MS Windows based programs that people run on their computers even just for spamming forums. Then programs on servers. And bot nets that run through virus infected computers/servers & are controlled remotely. There could even be people here reading this whose computer is part of a bot net and they don't even know it.

Dozens and dozens of bot spammers, the astrologer spam was probably the worst of it. We may have even had hundreds of automated spammers, not all posted in such huge amounts. They registered many thousands of accounts for posting spam. Some would register and not post spam for months or a year+ later. There were some bots that attempted to post normal messages, then after some time they would return & edit their messages to add in their spam links. Also spam bots that used to post links that were difficult to notice, semi-hidden. I also know of at least one instance where bots were able to guess an easy password on a GC account, went back and edited nearly a thousand messages just adding in various spam links.

In addition to the Indian astrology spam I recall some major fashion brand spamming GC for a long time as well. There was also a celebrity who must have hired spammers to promote her new show... I emailed them and told them to just sponsor GC to promote their show instead of spamming us.

Then my email... one account was receiving 8000+ emails a day when I decided to shut it down.

The spam on the forums and email was a real nightmare to deal with. May have been impossible to handle all that without the help of many site moderators.

GC also wasn't my only forum site. The other forums I shut down new account registrations on years ago. Was just too overwhelming to continue dealing with all the spam / spambots littering the sites with spam from every different direction.

On GC I made it difficult to easily create/verify new accounts and also use a crowd sourced spam detection service, which is quite imperfect and probably has too many false positives. That service has been used to block something like 10 million + account registrations on GC. I often wonder, though, about their false positive rate. Even 1% would be a huge loss. At some point I'll get around to looking into that more and will post additional details.

I don't know which is the most common way. There's surely a mix. Some are groups all knowing and working together. Others work together but do not know each other. And still others work alone or mostly alone.

There are definitely warehouses of people in other countries who are scamming people as their full time jobs. They go to work, catfish people in romance scams, etc. The vast majority of hackers attacking us here are not in this country.

How can scammers live with themselves--cheating people? What would their mothers think if they knew what they're really doing?

For the types I'm talking about, this is their job. They go to a big warehouse every day filled with computers and phones and go through their scams. It's a legit job in their countries. Sometimes sponsored by the government.

Bots are flooding GC again. Nearly 6,500 online users right now...

There are always bots, search spider bots are fine & not a problem. This current flood, though, is probably the AI scraper (data harvesting) type again.

Will try to get things sorted out with this batch of bots before they cause too much trouble...

Thank you for everything, John.

I thought you made it where it’d be harder for bots to attack the site. Did they manage to get around it?

The bots that were causing trouble today were sorted out earlier this evening. Highest I saw peak online hit today was just under 8,000.

:)

Partially, but not completely. It is harder for bots to continually cause trouble, but still plenty of room for improvement. When I spend more time working on GC projects again I'll button things down even more with regards to bot mitigation.

Sort of. We always have bots on GC. Some aren't programmed well, though, maybe due to software bugs, and they go a bit haywire while scraping or spidering sites. We're not blocking all bots but when they start causing problems I now at least have some ways of dealing with it. Imperfect solution that will be improved in the future.

Not sure yet but I've been thinking about spending more time in May and/or June on GC projects again, maybe improving bot mitigation will be included if I do that.

I noticed that the majority of them were looking at the same thread.

Yeah, it was an old thread, greeks in the family , only 14 pages... shouldn't take the bots 100+ thousand page views throughout the day to spider/scrape that thread. Somehow they sometimes get hung up and just keep going on the same thread.

Last summer was a similar issue. Bots from 2 IP addresses loading all the pages from the print version of the original "What do you feel like saying, right now?" thread. Those bots were loading something like 50 to 100 million pages per month from that thread. They were a major contributing factor causing the site to slow down so much for a while up until then.

The bot flood back in the fall I think was mostly scattered across many different pages.

They seemed to be gone or not as active for a while. Like, when you made those changes last summer, that seemed to work well.

We've hit a new record of bots on GC (unfortunately).

From the Online Users page:

"Most users ever online was 23274, Today at 08:10 AM."

That stat is over a 15 minute period. So 23,274 "users" from 7:55 to 8:10 AM today. Probably 99% bots.

The previous record was 15,051 back in August 2024.

Some bots are more difficult to deal with than others. Over time some are also getting better at disguising themselves, making it increasingly more difficult to manage.

The bots causing the massive slowdowns on GC leading up to last summer were originating from two specific IP addressed at a cloud web services provider. Those IPs I redirected in a way where it would not be clearly obvious that they were redirected and no longer on GC.

Since then I've redirected many other bots/IPs in a similar way.

Most seem to be originating from China, Singapore and now Brazil.

Not sure yet how I'll be managing this most recent onslaught of bots. The bulk of them seem to have gone away, but could be back at any moment.

Thank you so very much, John, for keeping GC up to date with the Battle Against the Bots!

Oh wow! How can you tell which countries they originate from?

John, where’s the coffee thread? I want to buy more coffee. You work so hard to keep this site going and I just want to thank you in the best way I possibly can.

You're welcome and thank you for helping to support the efforts with your GC subscription. :)

"Battle Against the Bots" for some reason reminded me of a TV show I used to watch some years ago called BattleBots: https://battlebots.com/ I used to love that show. Maybe I'll start watching again and when they're bashing each other to pieces I'll imagine they are the internet bots that were attacking GC. ;)

Usually when there's a number of similar IPs on GC that look suspicious I'll run a few through the search at https://www.arin.net/ and ban the entire IP range.

Sometimes the top level domain country code is in the IPs reverse DNS as well.

However, there's no guarantee that the actual source is from those countries. Could be they are just renting servers from datacenters / cloud server providers in those countries or those IPs could be from some sort of bulk proxy / VPN type services with IP addresses assigned to organizations in those countries.

I suppose it may also be possible that those IPs are registered to orgs in those countries but being used somewhere else.

:) Thanks, I appreciate it.

The original "Buy GreekChat a Coffee" thread is at https://greekchat.com/gcforums/showthread.php?t=248704

And the direct link to the Buy Me A Coffee page for GC is https://buymeacoffee.com/greekchat

Should we start referring to all of these bots now as Decepticons? Seems fitting.