brad's life - GMail Spam [entries|archive|friends|userinfo]
Brad Fitzpatrick

[ website | bradfitz.com ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

GMail Spam [Apr. 22nd, 2008|08:47 am]
Previous Entry Add to Memories Tell a Friend Next Entry
[Tags|, ]

GMail's anti-spam seems completely ineffective. This is what I woke up to this morning, after 8 hours of sleep:



7 messages I care about (only 1, really, but 6 I'll glance at), and 18 spams.

I was doing better when I was running my own mail server. :( If I look at the headers of these emails, a good number come from home DHCP users. My rule on my mail server, with a few false positives, was that sending email from IPs that reversed as home ISPs just wasn't allowed. Spam went to almost zero.

Also lols on how many of these messages have headers like:

X-IronPort-Anti-Spam-Filtered: true

Good job. I hope that header was faked and not actually IronPort's bragging.

What to do? I really hate mail.
LinkReply

Comments:
[User Picture]From: [info]dan_lane
2008-04-22 03:59 pm (UTC)

(Link)

I have a couple of friends who use spamarrest.com
[User Picture]From: [info]skx
2008-04-22 06:38 pm (UTC)

(Link)

Challenge-response systems are evil.

I run mail-scanning.com which is a potential solution for people, but even I'm nowhere near 100% effective.

Global stats show for the past 30 days total mail processed was 99% spam. And I know some gets through.

The arms race is constantly requiring adaption, and it is a pretty depressing process.

[User Picture]From: [info]octal
2008-04-22 03:59 pm (UTC)

(Link)

blacklisting has failed.

whitelisting is the Way Forward. There are market solutions, or just tmda. I'm doing tmda now, plus a good whitelist, plus spf.
From: [info]baudehlo
2008-04-22 04:06 pm (UTC)

TMDA is abuse

(Link)

Sorry, but can I PLEASE urge people not to use TMDA. The idea of spamming all the poor innocent spoofed From addresses in spam with your "did you send this?" emails is just WRONG. Braindead wrong. The author of TMDA should be ashamed of himself.
[User Picture]From: [info]octal
2008-04-22 04:12 pm (UTC)

Re: TMDA is abuse

(Link)

That's if you configure it incorrectly.

spf solves this.
From: [info]baudehlo
2008-04-22 04:17 pm (UTC)

Re: TMDA is abuse

(Link)

No it really doesn't. SPF solves very little. That's why many large sites that tried implementing SPF have had to roll it back.

Try this: dig txt earthlink.net
[User Picture]From: [info]skx
2008-04-22 06:38 pm (UTC)

Re: TMDA is abuse

(Link)

Hahahahahahahahahahah!
From: [info]seeds_of_peace
2008-04-22 06:01 pm (UTC)

Re: TMDA is abuse

(Link)

I'm curious what your view on BlueFrog was?
From: [info]baudehlo
2008-04-22 08:39 pm (UTC)

Re: TMDA is abuse

(Link)

Terrible idea. I work on a job (anti-spam) trying to STOP abuse of the internet. Flooding someone with connections, even if it's a spammer, is abuse. Pure and simple.

More importantly though it didn't stop spam. With fast flux hosting all it would do is flood some poor innocent user with a DSL line anyway.
[User Picture]From: [info]calebegg
2008-04-22 07:16 pm (UTC)

Re: TMDA is abuse

(Link)

Yes. Every few weeks I get hundreds of those, along with out of office replies and undeliverable messages. How are those supposed to be handled, spam-etiquette-wise? Should I mark them as spam in Gmail? One of my friends does that, but they're not really spam.

As for Gmail spam, it comes and goes for me. Today I got three or four in my inbox, but that's not very common. Gmail does pretty well for me. I have 3,668 in my spam box (so, in the last 30 days). My biggest issue with gmail's spam filters was about a year ago when I started getting a lot of CJK spam, which it completely failed at filtering for a long time.
[User Picture]From: [info]jwz
2008-04-22 05:59 pm (UTC)

(Link)

Oh man, fuck TMDA, srsly. If I mail (or worse, reply to) someone and I get a TMDA response, I just hit delete. I'm not jumping through hoops because you can't figure out how to manage filters.
[User Picture]From: [info]signe
2008-04-22 06:53 pm (UTC)

(Link)

Exacta-fucking-lutely. And I just love when people sign up on a site that has an automated mailing system (e.g. account validation) and then complain because they _didn't get the email_. (I'm postmaster at my org, so I tend to see many of these.)
[User Picture]From: [info]sandy
2008-04-22 04:01 pm (UTC)

Spam here too

(Link)

Yea, something must have changed with Google mail, because this is the first time I have ever gotten this stuff since moving our mail. I too woke up to a bunch, mostly foreign, junk mail.
[User Picture]From: [info]avva
2008-04-22 04:02 pm (UTC)

(Link)

It's doing much much better if messages are sent directly to your gmail account, than if they're forwarded. I don't know why (well, I know some of it, but it doesn't explain such a huge difference).
[User Picture]From: [info]brad
2008-04-22 04:34 pm (UTC)

(Link)

They're going straight to Google. Check my DNS.

And I verified in the headers that it's not going through mail.danga.com.
[User Picture]From: [info]e_richard
2008-04-22 04:04 pm (UTC)

(Link)

This is exactly why I stopped using Gmail after such a promising start.
From: [info]baudehlo
2008-04-22 04:08 pm (UTC)

Gmail's filters just aren't very good

(Link)

In fact I've always been confused why people think they are so great. I've always had LOADS of spam in my inbox on gmail.

But then I refuse to train it myself. You don't install an anti-virus product and expect to teach it what is a virus, so I don't expect to have to teach gmail what is spam.
From: [info]saccovanzetti
2008-04-22 04:47 pm (UTC)

Re: Gmail's filters just aren't very good

(Link)

I refuse to train it myself

How do you train it, other that pressing "Report spam" button instead of "Delete", which takes the same amount of effort ?

(Reporting spam really works in Gmail)
From: [info]baudehlo
2008-04-22 05:56 pm (UTC)

Re: Gmail's filters just aren't very good

(Link)

I never delete email.
From: [info]seeds_of_peace
2008-04-22 06:03 pm (UTC)

Re: Gmail's filters just aren't very good

(Link)

You never report spam? You never DELETE spam??
From: [info]baudehlo
2008-04-22 06:30 pm (UTC)

Re: Gmail's filters just aren't very good

(Link)

No. Reporting spam is very 1990s. It's useless now as it's all entirely botnet generated, and any ISPs that are hosting spammers either already know about it and are unresponsive, or are receiving reports from the likes of SpamCop anyway.

I don't delete spam because dealing with it is my job. I'm actually interested in keeping it :-)
From: [info]seeds_of_peace
2008-04-22 06:40 pm (UTC)

Re: Gmail's filters just aren't very good

(Link)

Well gmail's "report" function doesn't really get the purveyors in trouble, it just adds it to their spam mix so they can more easily prevent it in the future.. do you use gmail?
[User Picture]From: [info]calebegg
2008-04-22 07:25 pm (UTC)

Re: Gmail's filters just aren't very good

(Link)

It doesn't report it in the same way that SpamCop does, at least as far as I know. I assumed it used "Report Spam" to train the Bayesian filters. So if you don't report it, Gmail assumes you want other mail like it to go in your inbox. It's rather not like training an anti-virus program - more like training your dog to not pee on the carpet. By not reporting it, you're giving your dog a treat for peeing on the carpet.
From: [info]baudehlo
2008-04-22 08:36 pm (UTC)

Re: Gmail's filters just aren't very good

(Link)

I know what it does, but I don't agree that the analogy works. When you turn on the spam filters I create for my company they "just work". When I search on google it "just works", I don't have to train it how to find things. Spam isn't that hard to detect that I need to teach it how. Sorry, but google's filters are a VERY long way from the state of the art. On any given day the filters at my day job are about 99.75% accurate (we guarantee a 99.3% rate). That's with zero training whatsoever. Google should be able to achieve the same.
[User Picture]From: [info]fweebles
2008-04-22 04:42 pm (UTC)

(Link)

I just...don't have that problem with GMail. Looking at my inbox right now, I have zero spam. And I haven't moved anything out or filtered it automatically or anything.

Now, half of the email that comes to my GMail comes through a university alumni forwarding service that (I believe) runs spamassassin, but the other half comes straight in, so it's not like my GMail address isn't "out there" or anything.

I'm not sure what the difference is. I'm sure your e-mail address is more out there than mine, but still, I have 262 spam e-mails in my "Spam" Gmail folder and none in actual mail folders. Odd.
[User Picture]From: [info]brad
2008-04-22 05:10 pm (UTC)

(Link)

262 messages in your Spam email box? Nice. I have 15,474.
[User Picture]From: [info]topbit
2008-04-22 06:22 pm (UTC)

(Link)

In one of my accounts: "Spam (296945)" that's the full 30 days worth I guess. That account got moved onto Google's MX last summer when I was sick of running it on my own server, and now it mainly just forwards to my main Gmail account.

Like everyone else, there's been a lot more showing up in my various inboxes in the last few days though. Maybe a few dozen a day at the moment.
[User Picture]From: [info]lakeguy
2008-04-22 05:29 pm (UTC)

(Link)

I don't have any spam in my inbox either..

892 in my my "Spam" Gmail folder

only once every few months a spam message gets though
[User Picture]From: [info]syntheticjesso
2008-04-22 05:36 pm (UTC)

(Link)

Ditto this- all my spam goes to the spam folder. I can probably count on one hand the spam that's gone through to my inbox. I have about 100+ in there now I think, I get about that many every few days.
From: [info]Alex [tomkins.name]
2008-04-22 04:58 pm (UTC)

Very few problems here

(Link)

I must admit I did have one spam enter my inbox earlier today, but apart from that it seems to be highly effective at filtering.

My only problem with Gmail is that it tends to put certain mails from mailing lists into the spam folder.
[User Picture]From: [info]evan
2008-04-22 05:13 pm (UTC)

(Link)

I get a bunch of spam (less than you) but I've always blamed it on @danga.com forwarding confusing gmail. If you're not doing that, then I guess you're doomed. You could probably bring it up on an internal list.
[User Picture]From: [info]skx
2008-04-22 06:40 pm (UTC)

Re: Very few problems here

(Link)

It really does vary. I've got a couple of accounts there for testing purposes, and the filters manage to correctly file about 30,000 messages as spam a day.

On my personal gmail account I get about 3000 spam messages a day, and about 10 are let through into my inbox / mailing-list folders.
[User Picture]From: [info]avva
2008-04-22 07:27 pm (UTC)

(Link)

I'd echo that suggestion.
[User Picture]From: [info]phatduckk
2008-04-22 05:18 pm (UTC)

(Link)

I loath email. mostly due to the insane amount of SPAM I used to get. I switched my personal domain to use Yahoo's "personal address" service and things got a lot better... my spam went down from 20+ a day to about 6.

Fuzz's mail server's been getting a ton of spam lately too. It doesnt help that im on the distro list for everything (support, questions etc).
[User Picture]From: [info]dieplz
2008-04-22 05:32 pm (UTC)

(Link)

I get quite a lot of spam. In a typical week, only a handful of messages (2-3) get missed by the filtering, but in the last 24 hours I've had at least 10 or 15 make it through to my inbox. Either it was a busy Monday for spammers or somebody is messing with the filtering.
[User Picture]From: [info]docjeff
2008-04-22 05:35 pm (UTC)

(Link)

Best email service on the planet More filters than you can shake a stick at - really.
From: [info]cjensen
2008-04-22 05:37 pm (UTC)

(Link)

My non-gmail email address has only received five spams this month despite my address being all over the net since the early 1990s. I'm mystified as to what I'm doing right. I use imap clients which don't load images by default instead of webmail clients, which may be helping.
[User Picture]From: [info]infinitevoid
2008-04-22 07:04 pm (UTC)

(Link)

FYI: Gmail, while a webmail client, does not load images by default.
[User Picture]From: [info]martyshin
2008-04-22 05:42 pm (UTC)

(Link)

try www.mail.ru
i have no spam. really
From: [info]seeds_of_peace
2008-04-22 05:57 pm (UTC)

(Link)

I had about 6-7 sneak in.. looks like there's been a breakthrough in spam-bypass abilities
[User Picture]From: [info]mart
2008-04-22 07:39 pm (UTC)

(Link)

I seem to get periods every now and then where lots of spam gets into my Inbox on GMail, and then it calms down again. I've not got any spam in my Inbox for a few weeks now though, I think. I got a few thousand in my "Spam" folder in the last few days.

I'd love to know why it has these strange "off-periods".

[User Picture]From: [info]henry
2008-04-22 08:04 pm (UTC)

(Link)

I've been using http://www.spamato.net/ with Thunderbird for about a month and it's doing better than everything else I've tried. Much better than Thunderbird's built-in junk filter, better than gmail and waaay better than whatever Speakeasy's using.
[User Picture]From: [info]ghewgill
2008-04-22 08:40 pm (UTC)

(Link)

Gmail's spam filters seem to do better than a false negative rate of 10% for me. It's a bit hard to tell because most of my mail is filtered through spamhaus' zen list before being forwarded (which blocks like 95% of incoming mail), except for the spam that goes directly to my gmail acount.

You do have to train gmail's filter though. Both marking stuff as spam and trawling through the spam filter pulling non-spam out are required, in my experience. Doing the second part sucks, but it works for me.
[User Picture]From: [info]jon787
2008-04-22 10:04 pm (UTC)

(Link)

That header just confirms that it passed through the filter, not that it was spam.

Of course, IronPort has plenty of other issues. Including blatant SMTP violations that break so many things its not funny.
[User Picture]From: [info]shutterbc
2008-04-22 10:43 pm (UTC)

Different spam sources

(Link)

I couldn't help but notice the high amount of Russian spam you've got there.

I'm one of the relatively lucky ones that gets almost no false positives and about one spam message in the inbox per week. My spam folder has 4627 messages over the past 30 days.

What I'm wondering is if there are far fewer people training Gmail about the spam sources that have targeted you. There is a chance that a different set of spammers are targeting the US demographic vs. a Russian or Chinese or South American demographic. And I might wager you've dealt with more Russians (or Russian web sites) than me lately.

I'd have to know more about *how* competing mail services really filter spam before deciding that they'd be better or worse at it than gmail. Here's Google's not-so-technical explanation of how they filter spam (which to me sounds like bayesian filtering is involved at some level):
http://www.google.com/mail/help/fightspam/spamexplained.html

I just noticed that the chart they put up hasn't been updated since October of '07! Hope they're still updating the anti-spam parts anyway... :)
[User Picture]From: [info]wildilocks
2008-04-22 11:51 pm (UTC)

(Link)

I find Gmail to be very effective in trapping spam, in fact sometimes even too effective, as I've had the odd false positives, though they've been understandable - high volume automailed reports.

Gmail semi-regularly has spam "hiccups" though - I'd be surprised if your spam wasn't back to the usual amount within a couple of days at most.
[User Picture]From: [info]theducks
2008-04-23 12:29 am (UTC)

(Link)

Crazy option:

Stop using email

- Set up a bounce message pointing at an LJ post to receive messages on
- Include a phone number if people want to contact you urgently
- Read mailing lists via RSS

Problem solved

(and about 20 others created..)
[User Picture]From: [info]robflynn
2008-04-23 01:44 am (UTC)

(Link)

My spam count has increased the past few days as well. I was successfully blocking pretty much all the spam. I never had a false positive but I occasionally had a false negative. Now I get lots per day. Kind of like you, I had 5 emails I cared about, 29 that were spam. Ugh.
[User Picture]From: [info]xaotica
2008-04-23 02:16 am (UTC)

(Link)


weird, i see myself as not all that technically knowledgable, but skimming these responses i didn't see anyone mention the solution that i have.

dreamhost gives unlimited aliases, so i just create a new alias for anything i sign up for online. ie kim4nytimes, kim4yahoo, et al. they all route to the same inbox, and if i ever get spam i know exactly where it came from and can ream the person who left me open to it. no spam, no need for filtering, not really that much effort either.