Register Members List Search Today's Posts Mark Forums Read

Reply
 
Mod Options
Keyword weight based spam detector Details »
Keyword weight based spam detector
Mod Version: 0.1.0, by HuangA (Member) HuangA is offline
Developer Last Online: Jun 2013 I like it Show Printable Version Email this Page

This modification is in the archives.
vB Version: 3.7.0 Beta 5 Rating: (8 votes - 4.88 average) Installs: 38
Released: 08 Jul 2008 Last Update: 09 Jul 2008 Downloads: 197
Not Supported Uses Plugins Translations Is in Beta Stage  

I coded this one because I constantly had to moderate and / delete them lengthy lame cell phone ads on vBulletin.com's forums and my own forums. You know, buy iphone, ipod touch, noika blah blah blah sony ericsson blah blah blah etc. etc. etc. email us, we're legitimate business in a country you've never heard of, blah blah blah spam.

While Akismet does work on filtering them out, some times they still leak through.

I know there's two other keyword based tools that automatically adds things to moderation queue (One from SirAdrian and one from tweakmonkey), but it doesn't work too well for me, because I run an iPhone / iPod Touch site and I can't have those keywords on auto spam for simply appearing. So, here's what I did for mine...

What does this product do?
  • Adds 1 vBulletin Options setting group, with 4 settings
  • Allows you to define a list of keywords with associated score
  • Allows you to set a threshold for automatic moderation
  • Allows you to set a threshold for automatic rejection
  • Allows you to set a post count limit for posts to be scanned
  • Adds 1 plugin which gets ran at newpost_process
  • Adds 1 plugin which gets ran at editpost_update_process

How does it work?
1) You configure your keyword list, and score weight. For example, I use this list:

Block Disabled:      (Update License Status)  
Suspended or Unlicensed Members Cannot View Code.

The list basically means each time the plugin sees "Noika", it will get a score of 0.5, $, 0.5, etc. etc. A tally of all the score is totaled, and
2) You configure your moderation score, for example, I use 50.
3) You configure your rejection score, for example, I use 100.
4) You configure your exemption post count, for example, I use 5.

When a new post is being created (this could be a thread, or a reply, doesn't matter, they both trigger newpost_process hook), the plugin will count how many times each keyword appears, and total the score. If it is higher than or equal to the moderation score, it will tuck the post into moderation queue. If it is higher than or equal to the rejection score, a standard vBulletin error message is shown to the user.

How much overhead does this add? Realistically, not much... depending on amount of keywords used, I'd say most likely under 0.05 seconds of your CPU time for each post. If you are really that worried, you can set your exemption post count to something lower, and so lesser posts are scanned. Default is 5 right now.

This have been tested on 3.7.0 Beta 5, and 3.7.2. I see no reason why it would not work on 3.6.x series, too.

Change log
0.0.0 => 0.1.0
  • Changed error message to use vBulletin error message screen instead of die()
  • Added option for omitting after certain post count (default 5)
  • Added default values to options
  • Fixed options not appearing after product import (I forgot to export them for 0.0.0)
  • Added scanning for editing post (AJAX doesn't seem to give error... I'll work on that for 0.1.1 later)

Download Now

Only licensed members can download files, Click Here for more information.

Show Your Support

  • To receive notifications regarding updates -> Click to Mark as Installed.
  • If you like this modification support the author by donating.
  • This modification may not be copied, reproduced or published elsewhere without author's permission.
Similar Mod
Mod Developer Type Replies Last Post
Administrative and Maintenance Tools Multiple account login detector (AE Detector) MPDev vBulletin 3.6 Add-ons 598 25 Nov 2013 03:28

Comments
  #2  
Old 08 Jul 2008, 22:19
HuangA's Avatar
HuangA HuangA is offline
 
Join Date: Jun 2004
Real name: Andy Huang
<Reserving second post in thread, in case if I ever need to extend beyond the first post>
Reply With Quote
  #3  
Old 09 Jul 2008, 00:20
KURTZ KURTZ is offline
 
Join Date: Nov 2006
Real name: Christian
interesting Andy ... but just a question runs onto the latest vB?
Reply With Quote
  #4  
Old 09 Jul 2008, 00:25
youradhere4222 youradhere4222 is offline
 
Join Date: Sep 2007
This is fantastic! I've installed all of the keyword-moderation hacks but I've been having problems with effectiveness. Is there any way you could set a post count threshold for checking keywords? Also, does this work for edited posts as well?
Reply With Quote
  #5  
Old 09 Jul 2008, 00:44
HuangA's Avatar
HuangA HuangA is offline
 
Join Date: Jun 2004
Real name: Andy Huang
Originally Posted by KURTZ View Post
interesting Andy ... but just a question runs onto the latest vB?
I see no reason why it would not work with it. Though, I don't have a test forum to install it on. I'll try to work out a test forum tonight.

Originally Posted by youradhere4222 View Post
This is fantastic! I've installed all of the keyword-moderation hacks but I've been having problems with effectiveness. Is there any way you could set a post count threshold for checking keywords? Also, does this work for edited posts as well?
It doesn't work for edited posts yes. So in theory they can make a post with 10 characters first, and then edit it. I am planning to add that in to a later version to stop that work around.
Reply With Quote
  #6  
Old 09 Jul 2008, 01:21
Q-v-n-s-Q Q-v-n-s-Q is offline
 
Join Date: Mar 2005
Reserving, thank you
__________________
PossibleAndroid
Reply With Quote
  #7  
Old 09 Jul 2008, 06:26
HuangA's Avatar
HuangA HuangA is offline
 
Join Date: Jun 2004
Real name: Andy Huang
Apologies for the first person to install... If you gotten 0.0.0 instead of 0.1.0, please upgrade... it is probably best if you remove 0.0.0 and then install 0.1.0 because I changed the plugin name (for differentiation) and added the missing options (forgot to export them in first build and didn't notice it).

Aside from that, I did the post count thing so it only scans for a configurable amount of posts, and made it use error message screen instead of boring die() screen as per requested.

So in summary:
KURTZ: Yes, it works for 3.7.2
youradhere4222: Yes, it works for edit now (please install 0.1.0)
Reply With Quote
  #8  
Old 15 Jul 2008, 12:15
cheat-master30's Avatar
cheat-master30 cheat-master30 is offline
 
Join Date: Mar 2007
Location: Information Classified
Real name: cheat-master30
I think I might try this, because it might block some annoying spamming that I've seen without causing the disruption of censoring it.
__________________
Proud vBulletin supporter (cheat-master30 at official forum)
DS Ultimate- A Great Nintendo DS forum-
My Nintendo DS forum covering Mario Kart DS, Super Mario 64 DS and the like. Powered by the amazing vBulletin 3.7 software.
Reply With Quote
  #9  
Old 22 Jul 2008, 20:55
youradhere4222 youradhere4222 is offline
 
Join Date: Sep 2007
This works great!

This is somewhat of a long-shot suggestion, but in addition to having posts automatically rejected could we have users automatically banned for a pre-defined period if they hit a certain number of keywords? Also, to ensure that the ban was accurate, could a PM be sent (or even better a thread posted in a "staff forum" - like reported PM's and infractions) saying that xxx has been banned for xx days for posting the following message [ quote ] nokia, ipod, etc. [ /quote ]

Thanks!
Reply With Quote
  #10  
Old 23 Jul 2008, 14:29
HuangA's Avatar
HuangA HuangA is offline
 
Join Date: Jun 2004
Real name: Andy Huang
Originally Posted by youradhere4222 View Post
This works great!

This is somewhat of a long-shot suggestion, but in addition to having posts automatically rejected could we have users automatically banned for a pre-defined period if they hit a certain number of keywords? Also, to ensure that the ban was accurate, could a PM be sent (or even better a thread posted in a "staff forum" - like reported PM's and infractions) saying that xxx has been banned for xx days for posting the following message [ quote ] nokia, ipod, etc. [ /quote ]

Thanks!
Personally, I don't want to do that on my forum because of the possibility of false positives when I'm not around, and I could potentially ban someone who is genuinely interested in my forum before they even make their first post. But, I can see usefulness of that in some other forums, so I can certainly look into coding that some time this weekend or whenever I have time... no guarentee as to when I can push that out though.
Reply With Quote
  #11  
Old 24 Jul 2008, 14:56
youradhere4222 youradhere4222 is offline
 
Join Date: Sep 2007
Originally Posted by HuangA View Post
Personally, I don't want to do that on my forum because of the possibility of false positives when I'm not around, and I could potentially ban someone who is genuinely interested in my forum before they even make their first post. But, I can see usefulness of that in some other forums, so I can certainly look into coding that some time this weekend or whenever I have time... no guarentee as to when I can push that out though.
I agree, but let's say you have a competing site: competingsite.com

If they were frequently spamming you, you could enter the keyword and other variations to automatically ban anyone who uses it. It could also be used to auto-ban those who use racial slurs or use words you prohibit in the rules.
Reply With Quote
  #12  
Old 24 Jul 2008, 19:20
HuangA's Avatar
HuangA HuangA is offline
 
Join Date: Jun 2004
Real name: Andy Huang
Yes, there are certainly benefits to it. In your described case though, I'd still take additional percautions. I have had people coming to my site and first thing thy said was something like:
I just found this site from google, comparing to <competitor site>, this is way better and easier to use. Thank you for making this possible!!
If you do add competitor site to your keyword list, I'd recommend giving it some flexibility (ie: allow two occurances in post before it trigger moderation, and three or so before it trigger reject).

As mentioned, I'll look into coding an auto ban level during the weekend coming up, and update this again

PS: I'm considering a further "profile" system where we can create different sets of keywords/weights, so we can target spam better; but one problem I can see is if we add too many sets of profiles, the math required will probably take more CPU time... Any opinions on this, anyone?
Reply With Quote
  #13  
Old 29 Jul 2008, 11:11
HuangA's Avatar
HuangA HuangA is offline
 
Join Date: Jun 2004
Real name: Andy Huang
Sorry, just reporting in that I had a very busy weekend so I did not got around to work on this during the weekend. I will try to allocate some time aside this weekend for this.
Reply With Quote
  #14  
Old 17 Oct 2008, 09:25
veenuisthebest's Avatar
veenuisthebest veenuisthebest is offline
 
Join Date: Mar 2008
Real name: Vinayak
hello Andy..

This is one of the bestest spam preventing mods I have seen till now and it works perfect on my 3.7.3 PL1 board. Wonder why it has so less installs.

I think people like to stay away from mods that have a BETA tag to them. I hope you remove that BETA soon please

Thank you
Reply With Quote
  #15  
Old 18 Oct 2008, 06:57
HuangA's Avatar
HuangA HuangA is offline
 
Join Date: Jun 2004
Real name: Andy Huang
Thanks for the feedback, and sorry to everyone as I have not had a chance to update this because of development works... I have something similar (and hopefully even better) in the workings... stay tuned
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Mod Options

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


New To Site? Need Help?

All times are GMT. The time now is 21:23.

Layout Options | Width: Wide Color: