Register Members List Search Today's Posts Mark Forums Read

Reply
 
Thread Tools
  #1  
Old 30 Sep 2013, 03:45
abdobasha2004's Avatar
abdobasha2004 abdobasha2004 is offline
 
Join Date: Aug 2008
Sudden increase in server load at specific times

Within 6-11 pm GMT I get high load averages on my forum, my webhost says the load comes from showthread.php and MySQL.
I have tried disabling all mods but also load is very high.
My site is on a dedicated server.
This happens between 6-11 pm GMT when online are >13k.
Any other time the load is fine and the website is working fine.

url : nmisr.com
Any help?
Reply With Quote
  #2  
Old 30 Sep 2013, 07:57
Simon Lloyd's Avatar
Simon Lloyd Simon Lloyd is offline
 
Join Date: Aug 2008
Real name: Simon
You need to check your statistics..etc to see who is on your site at those times, which ip's and how many connections per ip.
__________________
Kind regards,
Simon Microsoft Office Help
My Mods: Find my modifications here
Please do not pm me for support unless i have invited you to!
Reply With Quote
  #3  
Old 30 Sep 2013, 08:11
CAG CheechDogg's Avatar
CAG CheechDogg CAG CheechDogg is offline
 
Join Date: Feb 2012
Lately there has been a rash of IPs hitting vbulletin sites for what ever reasons and most of them are Chinese IPs. I just helped another member here who was getting hit with tons of IPs and we blocked them through htaccess.

It is going to take a few hours of monitoring your site but you will get it under control. Here is the link to that member's thread :

http://www.vbulletin.org/forum/showthread.php?t=302628

If you need some help let me know, but it will require me to have access to your cPanel.
Reply With Quote
  #4  
Old 30 Sep 2013, 16:37
abdobasha2004's Avatar
abdobasha2004 abdobasha2004 is offline
 
Join Date: Aug 2008
thanks for help CAG and Simon, but all hacking or attacks possibilities are excluded
all the 13k online are random arab visitors !
Reply With Quote
  #5  
Old 30 Sep 2013, 17:18
CAG CheechDogg's Avatar
CAG CheechDogg CAG CheechDogg is offline
 
Join Date: Feb 2012
Ok good stuff..then just block all the ips from those areas and you should be good ...
Reply With Quote
  #6  
Old 30 Sep 2013, 18:59
Gripi Gripi is offline
 
Join Date: Jul 2009
It is very normal that MySQL and showthread.php consume lots of resource.

13k online in the same time? my site have 1200-1500 user online in the same time (showing from realtime google analytics), i'm using dual e2620 with memcache + apc, and 64gb ram, raid 10 ssd. the CPU load average is around 7.00 to 10.00.

As long as the site load time still fast, i think nothing to be worry about it.

Last edited by Gripi; 30 Sep 2013 at 19:05.
Reply With Quote
  #7  
Old 30 Sep 2013, 19:57
abdobasha2004's Avatar
abdobasha2004 abdobasha2004 is offline
 
Join Date: Aug 2008
Originally Posted by CAG CheechDogg View Post
Lately there has been a rash of IPs hitting vbulletin sites for what ever reasons and most of them are Chinese IPs. I just helped another member here who was getting hit with tons of IPs and we blocked them through htaccess.

It is going to take a few hours of monitoring your site but you will get it under control. Here is the link to that member's thread :

http://www.vbulletin.org/forum/showthread.php?t=302628

If you need some help let me know, but it will require me to have access to your cPanel.
Unbelievable !
When I added the code in htaccess the load markedly decreases, but it is still high about 50 with 14k online.
so do you have any updated list for these ips to block?
Reply With Quote
  #8  
Old 30 Sep 2013, 20:48
CAG CheechDogg's Avatar
CAG CheechDogg CAG CheechDogg is offline
 
Join Date: Feb 2012
Which one did you add the one from the thread I gave you?

Also use Simon's Ban Spiders by User Agent to block those bots we do not need on our forums and sites:

http://www.vbulletin.org/forum/showt...ht=ban+spiders

Here is the list I use, I block facebook, bing, msn, alexa and some others, why? I don't need them and they always seem not to play nice when they crawl my forums....I recommend you replace the list that comes with this mod with this one below:



almaden
Anarchie
Artabus
ASPSeek
attach
autoemailspider
BackWeb
Baidu
Bandit
BatchFTP
BlackWidow
Bot\mailto:craftbot@yahoo.com
Buddy
bumblebee
CherryPicker
ChinaClaw
CICC
Collector
Copier
Copyscape
Crescent
DIIbot
DISCo
DISCo\Pump
dotbot
Download\Demon
Download\Wonder
Downloader
Drip
DSurf15a
eCatch
EasyDL/2.99
EirGrabber
email
EmailCollector
EmailSiphon
EmailWolf
Express\WebPictures
ExtractorPro
EyeNetIE
FileHound
FlashGet
FrontPage
GetRight
GetSmart
GetWeb!
gigabaz
GNIP
Go\!Zilla
Go!Zilla
Go-Ahead-Got-It
gotit
Grabber
GrabNet
Grafula
grub-client
HMView
HTTrack
httpdown
.*httrack.*
ia_archiver
Ichiro
Image\Stripper
Image\Sucker
Indy*Library
Indy\Library
InterGET
InternetLinkagent
Internet\Ninja
InternetSeer.com
Iria
JBH*agent
JetCar
JOC\Web\Spider
JustView
larbin
LeechFTP
LexiBot
lftp
Link*Sleuth
likse
//Link
LinkWalker
Mag-Net
Magnet
Magpie
magpie
Mass\Downloader
Memo
Microsoft.URL
MIDown\tool
Mirror
Mister\PiX
Mozilla.*Indy
Mozilla.*NEWT
Mozilla*MSIECrawler
MS\FrontPage*
MSFrontPage
MSIECrawler
MSProxy
Navroad
NearSite
NetAnts
NetMechanic
NetSpider
Net\Vampire
NetZIP
NICErsPRO
Ninja
Nutch
Octopus
Offline\Explorer
Offline\Navigator
omgili
Openfind
PageGrabber
Papa\Foto
pavuk
pcBrowser
Ping
PingALink
Pockey
psbot
Pump
QRVA
RealDownload
Reaper
Recorder
ReGet
Scooter
Seeker
Siphon
sitecheck.internetseer.com
SiteSnagger
SlySearch
SmartDownload
Snake
sogou
Soso
SpaceBison
speedy
Spinn3r
sproose
Stripper
Sucker
SuperBot
SuperHTTP
Surfbot
Szukacz
tAkeOut
Teleport\Pro
URLSpiderPro
Vacuum
VoidEYE
Web\Image\Collector
Web\Sucker
WebAuto
[Ww]eb[Bb]andit
webcollage
WebCopier
Web\Downloader
WebEMailExtrac.*
WebFetch
WebGo\IS
WebHook
WebLeacher
WebMiner
WebMirror
WebReaper
WebSauger
Website
Website\eXtractor
Website\Quester
Webster
WebStripper
WebWhacker
WebZIP
Wget
Whacker
Widow
WWWOFFLE
x-Tractor
Xaldon\WebSpider
Xenu
Yandex
Yeti
YOUDAOBOT
Zeus.*Webster
Zeus
baiduspider
beta.statsit.com
statsit
SiteIntel
Yandex
GomezAgent
FunWebProducts
Nesotebot
DCPbot
AOL Advertising R&D
DataCha0s
aiHitBot
Apache-HttpClient
Zend_Http_Client
ReverseGet
XXX bot Content
vBSEO
spbot
OffByOne
thyroidbuzz
AcoonBot
coccoc
xpymep
proxyproxy2884
AppEngine
start.exe
Semiocast HTTP client
Firefox/3.6.23
TurnitinBot
curl
SwpLc/1.6
GrepNetstat.com
news bot
AskTbPTV
checks
panopta
App3le
PhantomJS
AlwaysOnline
SISTRIX
proximic
CRAWL-E/0.6.4
WebMoney
Maxthon
HTMLParser
oBot
UnisterBot
ERACrawler
Butterfly
Topsy
Butterfly Topsy Crawler
Ezooms
Deepnet
Alexa
Bitlybot
Seznam
Fulltext
Facebook
Sunrise Communications AG
crawl
Crawl
MJ12bot
Bimbot
Snapbot
thunderstone
Thunderstone
grub-client
Bing
MSN

--------------- Added 30 Sep 2013 at 20:50 ---------------

I also use this in my robots.txt file to block them before they even hit my forums:

User-agent: Baiduspider
Disallow: /

User-agent: Baidu
Disallow: /

User-agent: Baiduspider
Disallow: /

User-agent: Baiduspider-video
Disallow: /

User-agent: Baiduspider-image
Disallow: /

User-agent: Baiduspider-favo
Disallow: /

User-agent: Baiduspider-news
Disallow: /

User-agent: Baiduspider-mobile
Disallow: /

User-agent: Yandex
Disallow: /

User-agent: aipbot
Disallow: /

User-agent: ia_archiver
Disallow: /

User-agent: Alexibot
Disallow: /

User-agent: Aqua_Products
Disallow: /

User-agent: asterias
Disallow: /

User-agent: b2w/0.1
Disallow: /

User-agent: BackDoorBot/1.0
Disallow: /

User-agent: becomebot
Disallow: /

User-agent: BlowFish/1.0
Disallow: /

User-agent: Bookmark search tool
Disallow: /

User-agent: BotALot
Disallow: /

User-agent: BotRightHere
Disallow: /

User-agent: BuiltBotTough
Disallow: /

User-agent: Bullseye/1.0
Disallow: /

User-agent: BunnySlippers
Disallow: /

User-agent: CheeseBot
Disallow: /

User-agent: CherryPicker
Disallow: /

User-agent: CherryPickerElite/1.0
Disallow: /

User-agent: CherryPickerSE/1.0
Disallow: /

User-agent: Copernic
Disallow: /

User-agent: CopyRightCheck
Disallow: /

User-agent: cosmos
Disallow: /

User-agent: Crescent
Disallow: /

User-agent: Crescent Internet ToolPak HTTP OLE Control v.1.0
Disallow: /

User-agent: DittoSpyder
Disallow: /

User-agent: EmailCollector
Disallow: /

User-agent: EmailSiphon
Disallow: /

User-agent: EmailWolf
Disallow: /

User-agent: EroCrawler
Disallow: /

User-agent: ExtractorPro
Disallow: /

User-agent: FairAd Client
Disallow: /

User-agent: Fasterfox
Disallow: /

User-agent: Flaming AttackBot
Disallow: /

User-agent: Foobot
Disallow: /

User-agent: Gaisbot
Disallow: /

User-agent: GetRight/4.2
Disallow: /

User-agent: Harvest/1.5
Disallow: /

User-agent: hloader
Disallow: /

User-agent: httplib
Disallow: /

User-agent: HTTrack 3.0
Disallow: /

User-agent: humanlinks
Disallow: /

User-agent: IconSurf
Disallow: /
Disallow: /favicon.ico

User-agent: InfoNaviRobot
Disallow: /

User-agent: Iron33/1.0.2
Disallow: /

User-agent: JennyBot
Disallow: /

User-agent: Kenjin Spider
Disallow: /

User-agent: Keyword Density/0.9
Disallow: /

User-agent: larbin
Disallow: /

User-agent: LexiBot
Disallow: /

User-agent: libWeb/clsHTTP
Disallow: /

User-agent: LinkextractorPro
Disallow: /

User-agent: LinkScan/8.1a Unix
Disallow: /

User-agent: LinkWalker
Disallow: /

User-agent: LNSpiderguy
Disallow: /

User-agent: lwp-trivial
Disallow: /

User-agent: lwp-trivial/1.34
Disallow: /

User-agent: Mata Hari
Disallow: /

User-agent: Microsoft URL Control
Disallow: /

User-agent: Microsoft URL Control - 5.01.4511
Disallow: /

User-agent: Microsoft URL Control - 6.00.8169
Disallow: /

User-agent: MIIxpc
Disallow: /

User-agent: MIIxpc/4.2
Disallow: /

User-agent: Mister PiX
Disallow: /

User-agent: moget
Disallow: /

User-agent: moget/2.1
Disallow: /

User-agent: Mozilla/4.0 (compatible; BullsEye; Windows 95)
Disallow: /

User-agent: MSIECrawler
Disallow: /

User-agent: NetAnts
Disallow: /

User-agent: NICErsPRO
Disallow: /

User-agent: Offline Explorer
Disallow: /

User-agent: Openbot
Disallow: /

User-agent: Openfind
Disallow: /

User-agent: Openfind data gatherer
Disallow: /

User-agent: Oracle Ultra Search
Disallow: /

User-agent: PerMan
Disallow: /

User-agent: ProPowerBot/2.14
Disallow: /

User-agent: ProWebWalker
Disallow: /

User-agent: psbot
Disallow: /

User-agent: Python-urllib
Disallow: /

User-agent: QueryN Metasearch
Disallow: /

User-agent: Radiation Retriever 1.1
Disallow: /

User-agent: RepoMonkey
Disallow: /

User-agent: RepoMonkey Bait & Tackle/v1.01
Disallow: /

User-agent: RMA
Disallow: /

User-agent: searchpreview
Disallow: /

User-agent: SiteSnagger
Disallow: /

User-agent: SpankBot
Disallow: /

User-agent: spanner
Disallow: /

User-agent: SurveyBot
Disallow: /

User-agent: suzuran
Disallow: /

User-agent: Szukacz/1.4
Disallow: /

User-agent: Teleport
Disallow: /

User-agent: TeleportPro
Disallow: /

User-agent: Telesoft
Disallow: /

User-agent: The Intraformant
Disallow: /

User-agent: TheNomad
Disallow: /

User-agent: TightTwatBot
Disallow: /

User-agent: toCrawl/UrlDispatcher
Disallow: /

User-agent: True_Robot
Disallow: /

User-agent: True_Robot/1.0
Disallow: /

User-agent: turingos
Disallow: /

User-agent: TurnitinBot
Disallow: /

User-agent: TurnitinBot/1.5
Disallow: /

User-agent: URL Control
Disallow: /

User-agent: URL_Spider_Pro
Disallow: /

User-agent: URLy Warning
Disallow: /

User-agent: VCI
Disallow: /

User-agent: VCI WebViewer VCI WebViewer Win32
Disallow: /

User-agent: Web Image Collector
Disallow: /

User-agent: WebAuto
Disallow: /

User-agent: WebBandit
Disallow: /

User-agent: WebBandit/3.50
Disallow: /

User-agent: WebCapture 2.0
Disallow: /

User-agent: WebCopier
Disallow: /

User-agent: WebCopier v.2.2
Disallow: /

User-agent: WebCopier v3.2a
Disallow: /

User-agent: WebEnhancer
Disallow: /

User-agent: WebSauger
Disallow: /

User-agent: Website Quester
Disallow: /

User-agent: Webster Pro
Disallow: /

User-agent: WebStripper
Disallow: /

User-agent: WebZip
Disallow: /

User-agent: WebZip
Disallow: /

User-agent: WebZip/4.0
Disallow: /

User-agent: WebZIP/4.21
Disallow: /

User-agent: WebZIP/5.0
Disallow: /

User-agent: Wget
Disallow: /

User-agent: wget
Disallow: /

User-agent: Wget/1.5.3
Disallow: /

User-agent: Wget/1.6
Disallow: /

User-agent: WWW-Collector-E
Disallow: /

User-agent: Xenu's
Disallow: /

User-agent: Xenu's Link Sleuth 1.1c
Disallow: /

User-agent: Zeus
Disallow: /

User-agent: Zeus 32297 Webster Pro V2.9 Win32
Disallow: /

User-agent: Zeus Link Scout
Disallow: /

User-agent: Bimbot/1.0
Disallow: /

User-agent: Snapbot/1.0
Disallow: /

User-agent: snap.com beta crawler v0
Disallow: /

User-agent: magpie-crawler
Disallow: /

User-agent: MJ12bot
Disallow: /

User-agent: thunderstone
Disallow: /

User-agent: grub-client
Disallow: /
Reply With Quote
  #9  
Old 09 Oct 2013, 00:44
abdobasha2004's Avatar
abdobasha2004 abdobasha2004 is offline
 
Join Date: Aug 2008
unfortunately problem is not yet solved
Reply With Quote
  #10  
Old 09 Oct 2013, 00:52
ozzy47's Avatar
ozzy47 ozzy47 is offline
 
Join Date: Jul 2009
Real name: Chris
How many users are online at that time, following what was posted above?
__________________
You can get access to my 180 mods for vB 3.6 - 4.x at The Admin Zone as well as the professional support you are used to. New vBulletin Spider Definitions, vBulletin Spiders List Hits 1000 Spiders! ​ OzzModz down. Site has had a data breach, checking how the intrusion happened. Change your PW if you use the same one on my site and others.
Reply With Quote
  #11  
Old 09 Oct 2013, 01:12
Simon Lloyd's Avatar
Simon Lloyd Simon Lloyd is offline
 
Join Date: Aug 2008
Real name: Simon
4,571 right now!
__________________
Kind regards,
Simon Microsoft Office Help
My Mods: Find my modifications here
Please do not pm me for support unless i have invited you to!
Reply With Quote
  #12  
Old 09 Oct 2013, 01:16
ozzy47's Avatar
ozzy47 ozzy47 is offline
 
Join Date: Jul 2009
Real name: Chris
Well that should not be to bad then?
__________________
You can get access to my 180 mods for vB 3.6 - 4.x at The Admin Zone as well as the professional support you are used to. New vBulletin Spider Definitions, vBulletin Spiders List Hits 1000 Spiders! ​ OzzModz down. Site has had a data breach, checking how the intrusion happened. Change your PW if you use the same one on my site and others.
Reply With Quote
  #13  
Old 09 Oct 2013, 01:37
abdobasha2004's Avatar
abdobasha2004 abdobasha2004 is offline
 
Join Date: Aug 2008
How many users are online at that time, following what was posted above?
now about 4k but the problem happens when online are 13k or more

Well that should not be to bad then?
what should not be bad?!
Reply With Quote
  #14  
Old 09 Oct 2013, 01:43
ozzy47's Avatar
ozzy47 ozzy47 is offline
 
Join Date: Jul 2009
Real name: Chris
4K should not be bad, so adding all the above suggestions, and you still get 13k or more?
__________________
You can get access to my 180 mods for vB 3.6 - 4.x at The Admin Zone as well as the professional support you are used to. New vBulletin Spider Definitions, vBulletin Spiders List Hits 1000 Spiders! ​ OzzModz down. Site has had a data breach, checking how the intrusion happened. Change your PW if you use the same one on my site and others.
Reply With Quote
  #15  
Old 09 Oct 2013, 01:52
abdobasha2004's Avatar
abdobasha2004 abdobasha2004 is offline
 
Join Date: Aug 2008
yes, 95% are normal users not bots and at that time load increase exponentially !

--------------- Added 09 Oct 2013 at 01:53 ---------------

I have a big database with big posts table, should this table be partitioned?
Reply With Quote
Reply



Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


New To Site? Need Help?

All times are GMT. The time now is 16:29.

Layout Options | Width: Wide Color: