PDA

View Full Version : [How to] Cut down on memory usage


Brad
06 Jan 2006, 22:07
This is in the manual but I've yet to see it mentioned here.

The plugin system works by storing all plugin code for all scripts in memory, so you can quickly find your plugins using large amounts of memory if they contain a lot of code.

A simple way to avoid this problem is to use the plugin code simply to call an external script, which contains all the complex code. In this way the code is only loaded when it is actually required.

For example, a plugin could contain this:
$tmp_uid =& $vbulletin->userinfo['userid'];

$db->query_write("
INSERT INTO " . TABLE_PREFIX . "profilelog
(userid, dateline)
VALUES
($tmp_uid, " . TIMENOW . ")
");

or alternatively, that code could be placed into a file called (for example) plugins/my_script.php, and the plugin itself would contain this:
include('./plugins/my_script.php');

Naturally, the second option will use up far less memory than the first, and this saving will become more and more beneficial as the amount of code to be run increases.

You should also take a look at this project: http://www.vbulletin.org/forum/showthread.php?t=107315

Paul M
06 Jan 2006, 23:27
Would this slow execution if lots of plugins were calling lots of files ?

The Geek
06 Jan 2006, 23:38
Would this slow execution if lots of plugins were calling lots of files ?

A lot less then housing all the PHP for all files in memory at all time.

The bulk of my processing is done via external files.

Milad
03 Feb 2006, 08:24
Would this slow execution if lots of plugins were calling lots of files ?

I agree

I think files modifications would be better for this purpose

Marco van Herwaarden
03 Feb 2006, 09:28
I have been thinking about creating an extension that will allow 'include-files' for plugins. They would be loaded as regular plugins (not file uploads), and can be loaded into memory whenever needed (for example: require_pluginlibrary_once('myfunctionlibraryplugin'); ).

Like this you might get the best of both worlds:
- No file uploads
- Only loaded into memory when needed

PennylessZ28
03 Feb 2006, 18:26
I like that concept, I think I am going to play with this some more.

99SIVTEC
03 Feb 2006, 18:30
i'm ditching the plugin system on most of my larger sites. I'm just hardcoding the hacks. The plugin system is great in principal, but it eats up memory, increases queries, and slows down execution.

PennylessZ28
03 Feb 2006, 18:36
i'm ditching the plugin system on most of my larger sites. I'm just hardcoding the hacks. The plugin system is great in principal, but it eats up memory, increases queries, and slows down execution.

Oh so true. If only there was a way to use this concept of the xml plugin to also upload a file like the above. So its not stored in memory.

Marco van Herwaarden
03 Feb 2006, 20:37
i'm ditching the plugin system on most of my larger sites. I'm just hardcoding the hacks. The plugin system is great in principal, but it eats up memory, increases queries, and slows down execution.
If coded for saving memory(ie. 99% of executable code in include files), a plugin don't need to use much memory, nor does it need to use more queries or slow down significantly.

amykhar
03 Feb 2006, 20:40
This is in the manual but I've yet to see it mentioned here.

I feel better now. Somebody griped when I wrote my Amazon plugin this way :D

99SIVTEC
03 Feb 2006, 21:06
Just having the plugin system turned on uses extra queries. A good programmer ca hack vbulletin in such a way as to add few if any extra queries, but the system itself is what adds the extra queries.

If coded for saving memory(ie. 99% of executable code in include files), a plugin don't need to use much memory, nor does it need to use more queries or slow down significantly.

Marco van Herwaarden
03 Feb 2006, 21:10
Just having the plugin system turned on uses extra queries. A good programmer ca hack vbulletin in such a way as to add few if any extra queries, but the system itself is what adds the extra queries.

Not sure on the exact amount of added queries when the plugin system is turned on, but it is not much, and they are good optimiyed and won't effect your board much from a performance point of view.

Remember that the number of queries is almost irrelevant for performance, it is the execution time that counts.

PennylessZ28
03 Feb 2006, 21:21
Seems to me that just putting the plugin code in a php file and replacing it with

include('./plugins/my_script.php');

Doesn't work 100%

Marco van Herwaarden
03 Feb 2006, 21:45
Nobody ever said that things would work without any modification.

Andreas
03 Feb 2006, 21:57
1. Turning the Plugin System On or Off does not have any impact on the amout of queries.
2. Memory Footprint is not a such a big issue. In most cases, forumcache (which is loaded on every page) for example will be a lot bigger then the plugins.

Having to eval() the code all the time, that is an issue.

Therefore, complex plugin code (especially in high-traffic places like global.php, postbit, showthread, forumdisplay, etc.) should be moved out to include files.
(Could even be done automatically. I wrote a proof-of-concept hack that writes include-files if the plugin-code on one hook is > X bytes long.)

Andreas
03 Feb 2006, 21:59
Seems to me that just putting the plugin code in a php file and replacing it with

include('./plugins/my_script.php');

Doesn't work 100%

Hmm ... what problems did you have?

If you use

include(DIR . '/plugins/my_script.php');


it should work 100%.

PennylessZ28
04 Feb 2006, 01:17
Hmm ... what problems did you have?

If you use

include(DIR . '/plugins/my_script.php');


it should work 100%.

I converted two plugins to test this. VBSHOUT for one, which failed 100%.

And then estakis referral stats. That one works, half way.

It's depositing all the code in raw format on the page.

Crazy I tell ya.

Andreas
04 Feb 2006, 01:22
Erm ... are you sure you put


<?php
// Plugin Code here
?>


in the files?

PennylessZ28
04 Feb 2006, 01:57
Erm ... are you sure you put


<?php
// Plugin Code here
?>


in the files?

Yes I'm stupid. I forgot that. LMAO

Trigunflame
09 Feb 2006, 22:20
Solution to all your problems :)

http://www.vbulletin.org/forum/showthread.php?t=107315

pmfp
08 Apr 2006, 22:43
I have a problem with include's and phrases. It has been recreated in the files test.php and includes/test_include.php.

The problem is that the code is executed and the template fetched, but the phrase is not being printed. The nature of the real script means that I cannot use another plugin to insert the code, so that is out of the question.
Any ideas on how to do this properly?

test.php:
<?php

error_reporting(E_ALL & ~E_NOTICE);

define('THIS_SCRIPT', 'db_update');
$phrasegroups = array('test_phrases');

include('includes/test_include.php');

require_once('./global.php');

did_it();
?>
includes/test_include.php:

<?php

function did_it() {
$varvar = "set";

eval('print_output("' . fetch_template('test_splitter') . '");');
}
?>
test_splitter template:
<tr>
<td width="310" valign="top" class="thead">
<if condition="$varvar==set">$vbphrase[test_hello_world]</if>
</td>
<td valign="top" class="thead"></td>
</tr>

thincom2000
11 Feb 2007, 08:37
I just added about 10 of my larger plugins to files, and I have noticed a considerable lag on the forum. Page generation times have increased from .10 seconds to 2 seconds or more, and server load has gone up from about 1.00 to anywhere from 8 - 40. 40!!!!!

EDIT: Looks like it was just a really bad night for server load. My forum seems to be running at the normal load today.

Whissi
06 Jan 2008, 17:39
Do you still believe that outsourcing hugh plugins to includes will speed up your forum?
I don't think so. It sounds like it should improve the performance, but in real scenarios, it doesn't matter.

At least, have a look in the official addons like the blog software. They put their code in the plugins and not in files they include. They did it with high traffic hooks like fetch_userinfo... Haven't they thought about performance or doesn't that matter...