Register Members List Search Today's Posts Mark Forums Read

Reply
 
Thread Tools
  #706  
Old 12 Aug 2009, 05:55
amcd amcd is offline
 
Join Date: Oct 2004
I may have to run a cron job to check for any *.new.* files in that directory, and possibly put together a shell script to kill and restart searchd.
I would rather look for the cause of the malfunction. If most people do not get such problems, why should you?

--------------- Added 12 Aug 2009 at 05:57 ---------------

Think I'm safe to change "value" to "1" (and rebuild the datastore cache)?
I have no idea. Maybe someone from Jelsoft can answer. Or you can try reading the AdminCP PHP script which changes search type.
__________________
eXBii.com - Indian community
no XB no fun know XB know fun !

Last edited by amcd; 12 Aug 2009 at 05:57. Reason: Auto-Merged DoublePost
Reply With Quote
  #707  
Old 12 Aug 2009, 08:06
UK Jimbo's Avatar
UK Jimbo UK Jimbo is offline
 
Join Date: Sep 2002
Originally Posted by amcd View Post
I would rather look for the cause of the malfunction. If most people do not get such problems, why should you?
I agree on this one.

RedWingFan do you have a log of what the indexer is doing incuding any messages from the cronjobs?
__________________
administrator: visordown.com
Reply With Quote
  #708  
Old 13 Aug 2009, 19:10
Shamil. Shamil. is offline
 
Join Date: Sep 2008
Is this verified to work with the latest vB, 3.8.4 ?
Reply With Quote
  #709  
Old 16 Aug 2009, 19:12
RedWingFan's Avatar
RedWingFan RedWingFan is offline
 
Join Date: Oct 2004
Originally Posted by amcd View Post
I have no idea. Maybe someone from Jelsoft can answer. Or you can try reading the AdminCP PHP script which changes search type.
Well, I changed the value in the database, regenerated the options datastore, and it's working properly (as though I had chosen "Fulltext"). So that's a success!

Originally Posted by UK Jimbo View Post
RedWingFan do you have a log of what the indexer is doing incuding any messages from the cronjobs?
Memory allocation error during rotate. It happened again last night, and I found the memory allocation error in the log. I changed this value in my conf file: "seamless_rotate=0". Searching will be interrupted a bit when the indexes rotate, but since I do this during off-hours, it won't affect many users at all. (Apparently, Sphinx's indexer will load indexes into RAM so that users can still search while the old indexes are being replaced by the new.) The mem_limit is at 512MB right now, but I don't want to increase that and possibly starve everything else on the server. http://www.sphinxsearch.com/docs/cur...eamless-rotate

One other setting looks a bit deceptive: max_matches is set for 1000 in my conf file, but I'm only pulling in 500 in vB's search. What Sphinx does, for max_matches, is send back the BEST 1000 matches, not necessarily just running the search and returning the FIRST 1000 matches it finds. My point here is that I should be setting the maximum search results in the conf file and in vB to be the same number. Basically I'm searching for 1000 best matches, but throwing away 500 of them for visitors. I will probably bump the forum to display those 1000 matches. Visitors will think they're getting a bonus. http://www.sphinxsearch.com/docs/cur...nf-max-matches

Here is something else: http://www.sphinxsearch.com/docs/cur...nf-enable-star . You can search Sphinx using the asterisk ("star") as a wildcard. I have thought of enabling this, but I am thinking that the search_sphinx.php file (or vB itself) would strip out the asterisk and make no difference during searches. This would be a neat addition. Has anyone else here tried it?

I came across another option for the indexer. When indexing, there is a --merge option for the indexer, which will merge your delta indexes with your main indexes, rather than generating new indexes once per day. I have it running now where the indexes are regenerated once each night. Would there be any disadvantage to using merge? If it goes correctly, it should work just as well, since you have essentially the same indexes when you're finished. But I can also see a tiny opportunity for the main indexes to get corrupted. Otherwise, --merge takes less time and CPU cycles, which is attractive. http://www.sphinxsearch.com/docs/cur...#index-merging

Finally, I see that Sphinx also has a plugin for MySQL, where you can specify using SphinxSE as an additional engine in MySQL. It will not do us much good here, I know, but I could see a future use for it between vB and Sphinx. http://www.sphinxsearch.com/docs/current.html#sphinxse

IMHO, given how much Sphinx's popularity is growing, and after having pored over the documentation this afternoon, it is disappointing that there apparently will not be any built-in support for Sphinx in vB 4.0. Sphinx can do a lot, and is a lot more flexible, than the built-in vB search, as well as MySQL's own fulltext indexing. A shame we'll probably still have to patch these hacks together to use Sphinx...
__________________
-= N =-
Reply With Quote
  #710  
Old 17 Aug 2009, 05:42
amcd amcd is offline
 
Join Date: Oct 2004
I came across another option for the indexer. When indexing, there is a --merge option for the indexer, which will merge your delta indexes with your main indexes, rather than generating new indexes once per day. I have it running now where the indexes are regenerated once each night. Would there be any disadvantage to using merge? If it goes correctly, it should work just as well, since you have essentially the same indexes when you're finished. But I can also see a tiny opportunity for the main indexes to get corrupted. Otherwise, --merge takes less time and CPU cycles, which is attractive. http://www.sphinxsearch.com/docs/cur...#index-merging
moved/merged/edited/deleted posts are a huge reason to regenerate the whole index periodically.
__________________
eXBii.com - Indian community
no XB no fun know XB know fun !
Reply With Quote
  #711  
Old 17 Aug 2009, 16:35
RedWingFan's Avatar
RedWingFan RedWingFan is offline
 
Join Date: Oct 2004
Originally Posted by amcd View Post
moved/merged/edited/deleted posts are a huge reason to regenerate the whole index periodically.
Oh jeez...yeah, you're right! I didn't even think of that!
__________________
-= N =-
Reply With Quote
  #712  
Old 24 Aug 2009, 18:48
RedWingFan's Avatar
RedWingFan RedWingFan is offline
 
Join Date: Oct 2004
Originally Posted by RedWingFan View Post
Here is something else: http://www.sphinxsearch.com/docs/cur...nf-enable-star . You can search Sphinx using the asterisk ("star") as a wildcard. I have thought of enabling this, but I am thinking that the search_sphinx.php file (or vB itself) would strip out the asterisk and make no difference during searches. This would be a neat addition. Has anyone else here tried it?
Just curious if anyone has tried this at all. (This was buried in my last lengthy post.) I believe Sphinx also has phrase searching, where you enter search terms in quotes, but I have a feeling that vB strips out anything that's not alphanumeric...

Both of these would be a welcome addition to vB search.
__________________
-= N =-
Reply With Quote
  #713  
Old 09 Sep 2009, 20:47
columbusgeek's Avatar
columbusgeek columbusgeek is offline
 
Join Date: Sep 2007
RedWingFan are you using this on your music forum? Was trying to find examples of people using it, but having a hard time trying to find out if they are or not. Its a lot of work to get this installed on the forum, I want to make sure its worth it.
Reply With Quote
  #714  
Old 09 Sep 2009, 21:27
RedWingFan's Avatar
RedWingFan RedWingFan is offline
 
Join Date: Oct 2004
Yes, we're using it, and aside from a few minor quirks (which I can't even remember right now), it is working great! I don't know if it helped server loads or not (we had to do several changes at once to get us out of our overloads), but it returns searches much faster than using vB's own built-in search.

I still with vB would develop something official for this. The built-in searching (using vB or fulltext) is just too hard on the server resources...
__________________
-= N =-
Reply With Quote
  #715  
Old 10 Sep 2009, 07:24
amcd amcd is offline
 
Join Date: Oct 2004
Originally Posted by columbusgeek View Post
RedWingFan are you using this on your music forum? Was trying to find examples of people using it, but having a hard time trying to find out if they are or not. Its a lot of work to get this installed on the forum, I want to make sure its worth it.
Believe me, it is worth the trouble.
__________________
eXBii.com - Indian community
no XB no fun know XB know fun !
Reply With Quote
  #716  
Old 29 Sep 2009, 02:24
cobaku cobaku is offline
 
Join Date: Oct 2005
i somehow managed to set it up was difficult for me cause it was with latest stable version of sphinx

I have 1 problem i can only search latest data
is the problem this line
$cl->SetLimits(0, $vbulletin->config['Sphinx']['limit'], 1000);


indexer --config /etc/sphinx/sphinx.conf --all

did not work for me

do you see anything strange below ?


Block Disabled:      (Update License Status)  
Suspended or Unlicensed Members Cannot View Code.

is it normal nothing returns from postdelta and threaddelta

--------------- Added 29 Sep 2009 at 20:21 ---------------

if i can ever overcome this problem i will make a post sphinx search for dummies this post is so messed up.

Last edited by cobaku; 29 Sep 2009 at 20:21. Reason: Auto-Merged DoublePost
Reply With Quote
  #717  
Old 21 Oct 2009, 06:06
Raun Raun is offline
 
Join Date: Mar 2009
If I am installing this on a vbulletin installation that is setup as follows what box would it go on?

1load balancer
4 web servers
1 master DB
1 slave DB
Reply With Quote
  #718  
Old 21 Oct 2009, 06:35
DaiTengu DaiTengu is offline
 
Join Date: Dec 2005
Originally Posted by Raun View Post
If I am installing this on a vbulletin installation that is setup as follows what box would it go on?

1load balancer
4 web servers
1 master DB
1 slave DB
slave DB server would be your best bet.
Reply With Quote
  #719  
Old 21 Oct 2009, 08:22
UK Jimbo's Avatar
UK Jimbo UK Jimbo is offline
 
Join Date: Sep 2002
Originally Posted by Raun View Post
If I am installing this on a vbulletin installation that is setup as follows what box would it go on?

1load balancer
4 web servers
1 master DB
1 slave DB
It depends on the load on the two db servers. My guess would be that the least loaded in this configuration will be the master db server. Both master and slave are handling the same number of write (INSERT/UPDATE/DELETE) queries but the slave will be handling a load of reads too.

With a bit of scripting wrapping index copying (rsync probably) and restarting searchd you could generate the indexes on one machine and then copy them out to another one. This might be handy if you want to build some kind of redundancy into your setup or be able to balance sphinx traffic.
__________________
administrator: visordown.com
Reply With Quote
  #720  
Old 21 Oct 2009, 15:32
Raun Raun is offline
 
Join Date: Mar 2009
If I were to install it on both DBs would that work the same as copying the indexes?
Reply With Quote
Reply



Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


New To Site? Need Help?

All times are GMT. The time now is 14:59.

Layout Options | Width: Wide Color: