Mailing List Archive

>
> Do any of you use spamassassin with a gmail account, and if so, how are
> people doing it? The reason to do this is gmail's spam filtering isn't
> perfect

You can add to this, that gmail actually is also losing email and annoying is that you can't send zip files. I am constantly asking people to give me a different email address.


> We built some plumbing to do this using gmail's API, and also IMAP which
> can work with other services such as yahoo or outlook. I'm wondering if
> this is of any use to anyone other than myself.

I don't like any daemon connecting to my mail storage. Can you imagine if your solution gets hacked, how much data would be compromised? I prefer messages being scanned/marked before stored. I wonder if this is even gdpr compliant, because you can access private data constantly.


> Essentially, it's a daemon that connects to the account and acts as a
> mail client (an MUA). When messages arrive in a mailbox (could be any
> folder really), sucks out the message, runs it through spamassassin, and
> puts the result either into the Spam folder or Inbox.

Why not just forward messages? Register a domain put some mx servers in front of gmails mx. I recently was testing with such relay/forward, works perfectly, I am only changing the envelope nothing else. DKIM, spf everyting perfectly working.


> I'm just wondering what to do with this plumbing software, if it should
> be open sourced or run as a service. Running it as a service couldn't be
> free as I don't have access to free servers.

So for the whole of Europe you need data processing agreement for accessing the mail storage as a 3rd party.


> The daemon in it's current
> state is a bit complicated to set up on it's own but it could definitely
> be cleaned up, especially if there was sufficient interest.

I think this design is just wrong from the start. I have sometimes that we see that clients mailboxes are accessed from the digitalocean cloud because they granted access via their phone. Especially IOS is really insecure/bad with such privacy. It is just crazy giving access to your whole mailbox for maybe a 1 time action on a incoming email.


> I bet this could also be put together using getmail5 instead of this
> special built daemon but that would imply polling instead of push.
> Several ways to do this.
>

Maybe forget about this? ;)

From "Marc" <Marc@f1-outsourcing.eu>
>You can add to this, that gmail actually is also losing email and annoying is that you can't send zip files. I am constantly asking people to give me a different email address.
Yup! And it's not too difficult to pull messages out of the Spam folder
and put them back into Inbox. That is, if the message makes it into the
Spam folder and isn't rejected.

I don't know if it still works but I have had people send me zip files
to my gmail acct by renaming them as like .tip or .zap or something.
Frankly it's better to share such potentially large files in a link like
from dropbox, onedrive, or any one of a number of similar things.

>I don't like any daemon connecting to my mail storage. Can you imagine if your solution gets hacked, how much data would be compromised? I prefer messages being scanned/marked before stored. I wonder if this is even gdpr compliant, because you can access private data constantly.
First, for people like yourself, you would want to run such a daemon
yourself on your own infrastructure, hence why I am thinking of this
could be useful to other people as open source.

Second, there are plenty of people who don't run their own email, as in,
gmail users, that entrust their email to google. Though GDPR probably
has something to say about such a service, I doubt it would be
impossible under GDPR, especially EU users using a suitable EU server
and whatever rules necessary were followed.

>Why not just forward messages? Register a domain put some mx servers in front of gmails mx. I recently was testing with such relay/forward, works perfectly, I am only changing the envelope nothing else. DKIM, spf everyting perfectly working.
>
I'd be interested to know if anyone runs spamassassin forwarding from
gmail back into gmail, how does this work? How to get it so mail isn't
in a loop? You can't do what I'm talking about just by forwarding.
More below on that.

>So for the whole of Europe you need data processing agreement for accessing the mail storage as a 3rd party.
Probably, yes. Is it any different with a mail server that uses a back
end scanner as a service? I know there are several such services for
corporate email that work with a google workspace account that allows
you to modify the mail routing which you can't do with a free gmail
account.

>I think this design is just wrong from the start. I have sometimes that we see that clients mailboxes are accessed from the digitalocean cloud because they granted access via their phone. Especially IOS is really insecure/bad with such privacy. It is just crazy giving access to your whole mailbox for maybe a 1 time action on a incoming email.
I wouldn't say the design is ideal but I haven't seen any better way. I
didn't find a way to do it by forwarding myself, maybe I missed
something obvious? There's no way in consumer gmail to tell gmail to
loop messages through some external service. I guess you could forward
all messages and then use POP to "import" them back in. You wouldn't be
able to manipulate folders like the Spam folder or set up spam-training
and ham-training messages. I remain unconvinced just forwarding is the
best way to do this.

You can argue that it's really crazy giving access to your whole mailbox
to your email provider too. I guess I don't see the difference here.
Your mail service provider could be broken into as well. Read about
Microsoft's recent break-in?

I'm just wondering if there's enough interest in this to do the work to
make it open source. If there were a lot of people mailing me saying
"Yes! I've been looking for something like this but I don't want to run
it myself!", then I'd consider making it into a service, as well as
probably open sourcing it. Thing is, such a service has to minimally
viable. So far, you're the only response I've seen to this and your
response appears to be overwhelmingly negative.

In my own testing of this, my gmail Spam folder varies between 1500 and
5000 messages at any given time. Sometimes there's a false positive
that no matter how many times I tell gmail it's not spam, mail from that
user ends up in Spam. I also find gmail is not perfect and it misses
1-2 spams roughly every day that end up in my inbox. I have already
pressed the spam button once this morning. I've spent quite a bit of
time pulling down individual false negative messages and running them
through spamassassin on my server and they almost always get scored
highly as spam. So I personally find such a plumbing to be useful.

What I have is a plumbing that does the message manipulation and a bunch
of other things which are not pertinent. Some of the hard work is done,
it would still need some work to release to the world. Pulling messages
out and putting them back in is not as easy as it sounds and I can
honestly say the devil is in the details, but the good news is that part
now works well. I am just trying to figure out what to do with it, if
it's useful beyond family and friends, or if there is a more general
interest in being able to use spamassassin on other providers such as
gmail or yahoo. If there's insufficient interest, that's fine, I'll
just use it myself.

Michael Grant

On Mon, Apr 15, 2024 at 01:48:53PM +0000, Michael Grant via users wrote:
> > I don't like any daemon connecting to my mail storage. Can you imagine if your solution gets hacked, how much data would be compromised? I prefer messages being scanned/marked before stored. I wonder if this is even gdpr compliant, because you can access private data constantly.
> First, for people like yourself, you would want to run such a daemon
> yourself on your own infrastructure, hence why I am thinking of this could
> be useful to other people as open source.
>
> Second, there are plenty of people who don't run their own email, as in,
> gmail users, that entrust their email to google. Though GDPR probably has
> something to say about such a service, I doubt it would be impossible under
> GDPR, especially EU users using a suitable EU server and whatever rules
> necessary were followed.

Not impossible, no. But there are many things needed to implement
GDPR correctly, overhead is huge, and the fines are draconian, so I
wouldn't advise it unless you're willing to choose dealing with all
that as your main life career path. Not to mention that Google
themselves will likely block you (in better case) or sue you for ToS
violations before it could become financially viable model.

> > Why not just forward messages? Register a domain put some mx servers in front of gmails mx. I recently was testing with such relay/forward, works perfectly, I am only changing the envelope nothing else. DKIM, spf everyting perfectly working.
> >
> I'd be interested to know if anyone runs spamassassin forwarding from gmail
> back into gmail, how does this work? How to get it so mail isn't in a loop?
> You can't do what I'm talking about just by forwarding. More below on that.

I haven't really touched gmail in decade or few, but back then IIRC
it was relatively easy: you could choose to forward mail only when
some criteria was met (e.g. using email+extension@gmail.com, or some
header etc), instead of forwarding everything. And even if gmail no
longer supports that, you could implement loop handling on the other
side alone (just with a little more overhead)

> > So for the whole of Europe you need data processing agreement for accessing the mail storage as a 3rd party.
> Probably, yes. Is it any different with a mail server that uses a back end
> scanner as a service? I know there are several such services for corporate
> email that work with a google workspace account that allows you to modify
> the mail routing which you can't do with a free gmail account.

Well you'd likely need to hire a bunch of lawyers and study
requirements of GDPR for some months to model how it behaves in
corporate environment before engaging in risk assessment and building
your business model on top of those results.

> You can argue that it's really crazy giving access to your whole mailbox to
> your email provider too.

It *is* crazy. That's why all the cool kids ain't doing it for decades now.
They run their own VPS with SA, or install a FreedomBox or something. :)
Definitely don't depend on @gmail.com whatever !

> I guess I don't see the difference here. Your mail service provider
> could be broken into as well.

Sure, as can Gmail.. The difference is in statistics: even if such
service was technically and financially[1] as secure as Gmail (which
may be debatable), by the simple fact that your mail is now routed
through 2 SPOFs instead of just 1 SPOF means your chance of problems
has increased by at least 100% (i.e. doubled).

> I'm just wondering if there's enough interest in this to do the work to make
> it open source. If there were a lot of people mailing me saying "Yes! I've
> been looking for something like this but I don't want to run it myself!",
> then I'd consider making it into a service, as well as probably open
> sourcing it. Thing is, such a service has to minimally viable. So far,
> you're the only response I've seen to this and your response appears to be
> overwhelmingly negative.

Here is my advice: don't overthink it in advance. Instead:

- Pick a nice open copyleft FOSS license (e.g. AGPLv3+)
- write a dozen or so lines of most basic requirements and installation instructions in README
- publish whatever you have at the moment out on some source-hosting platform out there
(can't really recommend any really open one; you can self-host, or choose one of popular
ones, not really critical at this point)
- mention it at few related places

If people find it interesting, you'll note it in a number of issues,
feature requests, etc. As the demand grows, you can improve it.
If not, hey, you've wasted barely no effort, and did a good deed, so
it is a karma net positive in life, eh?

If it however turns out that it eventually becomes so popular so you
must choose between your day job and maintaining it, then you might
consider incorporating and launching it as a service. But not before.

> matter how many times I tell gmail it's not spam, mail from that user ends
> up in Spam. I also find gmail is not perfect and it misses 1-2 spams

"not perfect", seriously? Gmail is beyond hopeless, and has been for
more than a decade at least. Even your whole idea acknowledges that :)

> I am just trying to figure out what to do with it, if it's useful beyond
> family and friends, or if there is a more general interest in being able to
> use spamassassin on other providers such as gmail or yahoo. If there's
> insufficient interest, that's fine, I'll just use it myself.

I just opensource my stuff and put it out there. It's good practice.

> Michael Grant

[1] e.g. can you afford several teams of dozen people to monitor
everything 24/7 for anomalies? No? How big a bounty you have on
finding bugs and reporting them instead of exploiting them?

Well then, even if you use exact same techs, you're going to be
less reliable and easier to exploit then GMail. Only thing that
works in your favor is that you'll be less interesting for
targeted attacks, due to much smaller userbase.

--
Opinions above are GNU-copylefted.

> >Why not just forward messages? Register a domain put some mx servers in
> front of gmails mx. I recently was testing with such relay/forward, works
> perfectly, I am only changing the envelope nothing else. DKIM, spf
> everyting perfectly working.
> >
> I'd be interested to know if anyone runs spamassassin forwarding from
> gmail back into gmail, how does this work? How to get it so mail isn't
> in a loop? You can't do what I'm talking about just by forwarding.
> More below on that.

You have to get a domain and put in front. You need to be able to set your own mx records so you can do your scanning of messages on these mx servers. This is how most of these 'anti spam' providers work.


>
> In my own testing of this, my gmail Spam folder varies between 1500 and
> 5000 messages at any given time. Sometimes there's a false positive
> that no matter how many times I tell gmail it's not spam, mail from that
> user ends up in Spam.

I am actually suprised to read that. I have currently a setup where users can drag a message to a folder and than the sender is whitelisted for any future message.

> I also find gmail is not perfect and it misses
> 1-2 spams roughly every day that end up in my inbox. I have already
> pressed the spam button once this morning. I've spent quite a bit of
> time pulling down individual false negative messages and running them
> through spamassassin on my server and they almost always get scored
> highly as spam. So I personally find such a plumbing to be useful.

You have to also check if not a lot of spamassassin knowledge comes from external sources like dnsbl and dnsuribl. If you would scale your service, you need to start paying for these.

Register a new domain notgmail.com, setup your own mx forward, scan and forward to gmail. Afaik you should be able to configure gmail to use notgmail.com as outgoing email address.

> What I have is a plumbing that does the message manipulation and a bunch
> of other things which are not pertinent. Some of the hard work is done,
> it would still need some work to release to the world. Pulling messages
> out and putting them back in is not as easy as it sounds and I can

With sieve it is not that difficult. If a user drags it to a specific folder, it is unmarked and the unmarked message is put back in inbox and the sender is whitelisted for ever.

> honestly say the devil is in the details, but the good news is that part
> now works well. I am just trying to figure out what to do with it, if
> it's useful beyond family and friends, or if there is a more general
> interest in being able to use spamassassin on other providers such as
> gmail or yahoo. If there's insufficient interest, that's fine, I'll
> just use it myself.
>

If you like developing such stuff, you should look into this unified messaging/document storage. Lots of companies are interested in better archiving their documents with their correspondence. Most of those services hook into your mailbox to do all kinds of management tagging, searching etc. That has target audience willing to pay for such service.

Matija

Sorry, you have misunderstood what I posted. I am not at all advocating
people use gmail. Something like 68% of the planet already uses it and
few people like you and me have the skills to host our own email. It's
not crazy for the people who use gmail or yahoo or other providers, they
use it, they're used to it, and they apparently like it enough not to
leave.

It's not easy for people to run their gmail acct through spamassassin.
Maybe some hack with forwarding and adding headers and a check for
looping might work. This isn't what I was really talking about. But it
doesn't matter.

Michael Grant

Michael Grant via users skrev den 2024-04-15 12:55:
> Do any of you use spamassassin with a gmail account, and if so, how
> are people doing it? The reason to do this is gmail's spam filtering
> isn't perfect and you don't have the control you have with
> spamassassin.

...

https://isbg.gitlab.io/isbg/index.html

support gmail and spamassassin

other then that i tryed to make a gentoo ebuild for it, have to retry
now :)

>
>https://isbg.gitlab.io/isbg/index.html
>
>support gmail and spamassassin
>
>other then that i tryed to make a gentoo ebuild for it, have to retry now :)
>
Yes that's kinda similar! I'll have to try that! Thanks.

On 16/04/2024 08:24, Michael Grant via users wrote:

> I am not at all advocating people use gmail. Something like 68% of the
> planet already uses it and few people like

I really wonder about that, or did they pull a trump...
I ran this June last year, the results are somewhat surprising, of
course YMMV depending upon you're countries politics or what your ISP is
like I guess.
https://blog.ausics.net/archives/147-How-do-you-use-Email.html

> Michael Grant

--
Regards,
Noel Butler