PDA

View Full Version : [RELEASE] Usenet Gateway for vB 2.01


Pages : [1] 2 3

fastforward
12 Jul 2001, 07:18
Description:

This hack provides a gateway to selected usenet groups. The entire usenet group is mirrored locally. Posts submitted locally to mirrored newsgroups will also be sent out to usenet. A single perl script is executed via cron that collects news and posts any outgoing articles. Incoming messages are parsed against customizable spam filters and threaded correctly before being placed in your forums. By default, no personal information is sent to usenet other than the posters username and whatever you have globally configured as a footer and organization. The option does exist for a user to include a custom email to use for usenet posts. After initial installation (which involves creating some new tables, adding a few columns to existing vB tables, and entering the database connection information to the news script) configuration is completed via the vB control panel.

See it in action at dBforums.com (http://dbforums.com) (usenet forums are at the bottom)
Download (http://britishexpats.com/download/usenet_gateway.tar.gz) latest version for vB 2.2.4


Current version is 20020323 for vB2.xx

Current feature list:
correct threading of outgoing posts on usenet
full control panel integration
email notification to usenet replies
emoticon translation into vb icons
vb code removal or conversion prior to posting to usenet
hyperlinked urls in messages
color coded, italicized & indented quotes
vB style quote to usenet style quote conversions for outgoing posts
logging of outgoing posts
support for multiple news servers
support for seperate footers per forum in outgoing posts
multi-language (selectable) handling of quoted MIME printable headers (for all those funny foreign characters)
handling of mailing archive groups
flexible spam control and replacement variable options for incoming and outgoing messages
configurable auto-expire option
option for users to show email address in their outgoing usenet posts (may be different than normal one in profile
option to enable/disable user signatures on outgoing posts.
fully compatible with vB moderation functions

Still to be done:
canceling of messages after they have been sent to usenet via control message in accordance with RFC1036.
binary attachment support
email to PM gateway

Screeshots:
Main Options (http://britishexpats.com/download/Image1.gif)
More Options (http://britishexpats.com/download/Image4.jpg)
Configuration of Newsgroups (http://britishexpats.com/download/Image2.jpg)
Add New Newsgroups (http://britishexpats.com/download/Image5.jpg)
Spam Control (http://britishexpats.com/download/Image3.jpg)
What an imported usenet article looks like in vB (http://britishexpats.com/download/Image6.jpg)

The latest version of this hack will always be in this first post of the thread.

Before asking questions in this thread, please read all the posts in this thread. If your question goes unanswered, it will probably be because the question has already been answered countless times in this thread.

This hack was created for use on my forums and will only be supported as time permits.


Links to required Perl Modules News::NNTPClient (http://search.cpan.org/search?dist=NNTPClient)
Net::SMTP (http://search.cpan.org/search?dist=libnet)
MIME::WordDecoder (http://search.cpan.org/search?dist=MIME-tools)
Text::AutoFormat (http://search.cpan.org/search?dist=Text-Autoformat)
Mail::Address (http://search.cpan.org/search?dist=MailTools)
URI::Find (http://search.cpan.org/search?dist=URI-Find)
DBI (http://search.cpan.org/search?dist=DBI)
DBD::MySQL (http://search.cpan.org/search?dist=Msql-Mysql-modules)
Date::Parse (http://search.cpan.org/search?dist=TimeDate)
HTML::Entities (http://search.cpan.org/search?dist=HTML-Parser)

george_proost
12 Jul 2001, 11:29
Little Problem with SEARCH.PHP

The new code for ...newusenet search does not seem to work.

I get redirected to the standard search instead of displaying the no results forund.

Any help appreciated ... I'm still looking :)


==========================================

http://130.94.169.47/images/USENET.gif
download it (http://130.94.169.47/images/USENET.gif)    --- right click...save target as

orca
12 Jul 2001, 15:18
Great work. Installation went very smoothly. Binary attachments aren't that important for me. I'm still playing abit around.
A great feature would be, that Usenet posters would be given the title Usenet guest or so automatically.

george_proost
12 Jul 2001, 20:05
I seem to be having some trouble. I have upgraded as per the instructions. I seem to be loading quite a bit of data, but no population of the threads is observed. (sometimes... 1 or 2 pop up )

I also restarted from scratch some loads and after a while I get the following :


sh-2.04$ perl www/admin/newnews.pl
Connecting to news.newzpig.com... Connected
Sending authentication info... Authenticated and logged in
Processing outgoing posts
Getting article batch from bit.listserv.ispf-l
No new messages in bit.listserv.ispf-l
inserting new threads from bit.listserv.ispf-l
Processing outgoing posts
Processing outgoing posts
Getting article batch from bit.listserv.tsorexx
No new messages in bit.listserv.tsorexx
inserting new threads from bit.listserv.tsorexx
Processing outgoing posts
Processing outgoing posts
Getting article batch from comp.databases.ibm-db2
Fetching headers of articles 46866 to 46915...
I 46866-46915 = 49 ( I requested 50 ) does this mean that if I request 1, nothing will happen ?
yip request one get none
Processing headers for article 46867...
Where is 46866 ? Hmm leaves 48 now to process
Processing headers for article 46868...
Processing headers for article 46869...
Processing headers for article 46870...
Processing headers for article 46871...
Processing headers for article 46872...
Processing headers for article 46873...
Processing headers for article 46874...
Processing headers for article 46875...
Processing headers for article 46876...

<snip>

Processing headers for article 46892...
Processing headers for article 46893...
Processing headers for article 46894...
Processing headers for article 46895...
Processing headers for article 46896...
Processing headers for article 46897...
Processing headers for article 46898...
Processing headers for article 46899...
Processing headers for article 46900...
Processing headers for article 46901...
Processing headers for article 46902...
Processing headers for article 46903...
Processing headers for article 46904...
Processing headers for article 46905...
Processing headers for article 46906...
Processing headers for article 46907...
Processing headers for article 46908...
Processing headers for article 46909...
Processing headers for article 46910...
Processing headers for article 46911...
Processing headers for article 46912...
Processing headers for article 46913...
Processing headers for article 46914...
Processing headers for article 46915...
Fetching article body 46867... OK
Where is 46866 ? Hmm leaves 48 now to process
Fetching article body 46868... OK
Fetching article body 46869... OK
Fetching article body 46870... OK
Fetching article body 46871... OK
Fetching article body 46872... OK
Fetching article body 46873... OK
Fetching article body 46874... OK
Fetching article body 46875... OK
Fetching article body 46876... OK
<snip>
Fetching article body 46901... OK
Fetching article body 46902... OK
Fetching article body 46903... OK
Fetching article body 46904... OK
Fetching article body 46905... OK
Fetching article body 46906... OK
Fetching article body 46907... OK
Fetching article body 46908... OK
Fetching article body 46909... OK
Fetching article body 46910... OK
Fetching article body 46911... OK
Fetching article body 46912... OK
Fetching article body 46913... OK
Fetching article body 46914... OK
Fetching article body 46915... OK
Processing article batch...
Requested 49 messages... 0 not available or rejected.
I REQUESTED 50 actually records 48
inserting new threads from comp.databases.ibm-db2
Processing outgoing posts
Processing outgoing posts
Getting article batch from bit.listserv.asm370
No new messages in bit.listserv.asm370
inserting new threads from bit.listserv.asm370
Processing outgoing posts
Processing outgoing posts
Getting article batch from comp.lang.apl
No new messages in comp.lang.apl
inserting new threads from comp.lang.apl
Processing outgoing posts
Processing outgoing posts
Getting article batch from comp.lang.cobol
No new messages in comp.lang.cobol
inserting new threads from comp.lang.cobol
Processing outgoing posts
Processing outgoing posts
Getting article batch from comp.lang.fortran
No new messages in comp.lang.fortran
inserting new threads from comp.lang.fortran
Processing outgoing posts
Processing outgoing posts
Getting article batch from comp.lang.rexx
No new messages in comp.lang.rexx
inserting new threads from comp.lang.rexx
Processing outgoing posts
Processing outgoing posts
Getting article batch from comp.lang.c++.moderated
No new messages in comp.lang.c++.moderated
inserting new threads from comp.lang.c++.moderated
Processing outgoing posts
Clean disconnection from news.newzpig.com
Purging usenet articles...
Purging usenet threads older than 356 days from bit.listserv.ibm-main... OK
Purging usenet threads older than 356 days from bit.listserv.vse-l... OK
Purging usenet threads older than 356 days from bit.listserv.ispf-l... OK
Purging usenet threads older than 356 days from bit.listserv.tsorexx... OK
Purging usenet threads older than 356 days from comp.databases.ibm-db2... OK
Purging usenet threads older than 356 days from bit.listserv.asm370... OK
Purging usenet threads older than 356 days from comp.lang.apl... OK
Purging usenet threads older than 356 days from comp.lang.cobol... OK
Purging usenet threads older than 356 days from comp.lang.fortran... OK
Purging usenet threads older than 356 days from comp.lang.rexx... OK
Purging usenet threads older than 356 days from comp.lang.c++.moderated... OK
Purge completed.

george_proost
12 Jul 2001, 22:19
I needed to do this to the code to be able to understand it ..
it sorts out the counting problem
(fetchlimit=1 also works)
... the console output I added '+1' text to confirm...and now reads
Fetching headers of articles 46866+1 to 46915...

I still get NO threads populated
the above '+1' issue needs to be resolved.


post_outgoing_t($c,\$newsgroup);

my ($svrfirstmsg,$svrlastmsg,$tmp) = $c->group($newsgroup->{newsgroup});

if ($newsgroup->{lastmsg} eq 0) {
$newsgroup->{lastmsg} = $svrfirstmsg; }

elsif ($newsgroup->{lastmsg} > $svrlastmsg) {
$newsgroup->{lastmsg} = $svrlastmsg; }

$batch_end = $newsgroup->{lastmsg}+$config{batch_limit};

if ($batch_end > $svrlastmsg) {
$batch_end = $svrlastmsg; }

console("Getting article batch from $newsgroup->{newsgroup}\n");

if ($svrlastmsg > $batch_end) .......



I'm off to bed

fastforward
12 Jul 2001, 23:07
Those message numbers are not guaranteed to be incremental. Articles are requested from the server based on the current pointer within the group. The server will return the next available message number, NOT the next sequential message number.

As far as the other problem, hold off posting reams of logs and give me a chance to get home and have a look at it.

I'd advise everyone else to wait a few hours before installing... unless it's too late :)

george_proost
12 Jul 2001, 23:07
+notify

fastforward
12 Jul 2001, 23:16
Originally posted by george_proost
+notify
What's that supposed to mean?

Also, using big red text demanding that things need resolving doesn't really motivate me to go out of may to fix this.

fastforward
13 Jul 2001, 00:58
This release fixes one major bug and suppresses the new annoying feature of the new DBD::MySQL module.

v20010713 (for 2.01) (http://britishexpats.com/download/usenet_gateway.tar.gz)

Posts not getting loaded bug fix. This turned out to be just a misplacement of the next() method. The pointer was never moving to the next message because the code was placed outside the loop. :rolleyes:

Suppressed DBI dup key warning messages. I upgraded my modules and noticed a whole bunch of informational warning messages start to appear whenever a duplicate key was violated. The violations are supposed to happen. The script lets MySQL do the work of checking the constaints in the word table. This is far more efficient than trying to figure it out in code (especially on large tables). The messages have been suppressed and you don't need to know about them :). This is probably what some people were assuming to be errors. There was no need to worry and indexing need not have been disabled. :)

Fixed Option description in CP. The 'Suppress Usenet Posts in Stats' is back to front. Without a code change, they will be shown. With the code change, this option acts as a toggle.


If you installed v20010712 last night or today, you just need to replace the newnews.pl and run the SQL in the upgrade script to fix the control panel typo.

Download Here (http://britishexpats.com/download/usenet_gateway.tar.gz)

fastforward
13 Jul 2001, 05:17
Another couple of little bugfixes.

v20010714 (for 2.01) (http://britishexpats.com/download/usenet_gateway.tar.gz)

Fix to prevent duplicates being loaded into forums

Fix to stop hashes being displayed in place of angled brackets around address.


If you installed v20010713 today, you just need to replace the newnews.pl.

Download Here (http://britishexpats.com/download/usenet_gateway.tar.gz)

george_proost
13 Jul 2001, 08:29
Dear Paul (FF),

The intention is not to rattle your cage. I'd never do that.
All I wanted to do was make notes etc.. while I went along.
Be assured, you are doing a tremendously good job here.
I want to cooperate.

The red, ... yea is red ... for me outstanding issues
the green, things that seem fixed. due to my muddling around.

It's looking better everey day...

The sad part is that I have been told to move my server, I
apparently crashed a shared server out 3-4 times, a 'kernel panic' they said... whatever that means.

Any ideas for a good ISP.

orca
13 Jul 2001, 08:44
George, try http://www.venturesonline.com

I run the gateway there now for 3 days. No complaint yet. They even installed the modules for me without any problems. I only had to ask for them :).

george_proost
13 Jul 2001, 09:25
it is www.venturesonline.com that are dumping me ...

fastforward
13 Jul 2001, 14:56
HostPro/vservers are very good. I used their Freedom servers (semi dedicated) before I moved to a dedicated at Dialtone.

George... just ignore my outburst earlier. I was having a bad day at work (no change there) :) I apologize.

Has anybody installed the latest with bug fixes yet?

orca
13 Jul 2001, 17:54
I did and I don't seem to have any problem yet. Works smoothly. Well, I don't have console output on anymore. Used to have many "duplicate entry" if there were new posts.


Just checked, everything ok.

fastforward
13 Jul 2001, 18:04
Originally posted by orca
I did and I don't seem to have any problem yet. Works smoothly. Well, I don't have console output on anymore. Used to have many "duplicate entry" if there were new posts.
Those messages have now been suppressed. I wasn't aware that these were happening before. It must be specific to the Perl MySQL module version as I just upgraded and it started happening to me. They were never real errors though, just informational messages.

george_proost
13 Jul 2001, 18:14
is this a suitable config ?


Intel Server
866 MHZ Pentium III, 256KB cache
512 MB ECC RAM
20.5 GB, EIDE Hard Drive
Webhost Manager and Cpanel3
25 GB of Burstable Traffic


software
============
Apache 1.3.17
RedHat 6.2
PHP4.0.4PL1
MySQL Ver 11.12 Distrib 3.23.32, for pc-linux-gnu(i686)
Perl 5.6.0
Zlib
Pre-installed cgi scripts

It will only run 1 vBulletin

orca
13 Jul 2001, 19:18
At george: Yeah, it looks quite good.
It seems that newnews may cause server troubles because today I can't get to my site. No ping time and no trace possible...
But not sure whether it is that script...No mail from venturesonline yet.

george_proost
13 Jul 2001, 19:38
is the provider. Paul L has been a great help. I thank him for it.

george_proost
14 Jul 2001, 11:53
Hi All,
I move my domain to a new server etc....
Installed all required modules etc ....

I get :

bash-2.04$ perl newnews.pl
News::NNTPClient: Can't connect socket: Connection timed out at newnews.pl line 274
Connecting to news.newzpig.com... Connected
News::NNTPClient: News::NNTPClient::SOCK1 has been closed
at newnews.pl line 278
Authentication failure at newnews.pl line 278.
Sending authentication info....

I am a perl beginner .. :D

TIA

orca
14 Jul 2001, 14:21
George, it seems that you gave wrong login info to the news server.
The problem I had is resolved. It wasn't the script. Just ran it again and now problems anymore.

george_proost
14 Jul 2001, 14:33
I think I chaged something

news.xxxxxxx.com:????

I took the :???? off could that be it...
what was the port number anyway ?

orca
14 Jul 2001, 14:41
Port for news is usually 119, however, you may don't need to supply it because the script does it. If it would be missing, the error would rather be not connected than login failure. Be sure that you supply the correct username and password. If it's anonymous, you can leva the fields blank in most cases.

george_proost
14 Jul 2001, 15:42
doing test loads now ... all ok so far.

.. need to check the ... substitution somewhere

orca
14 Jul 2001, 16:27
Hmm, never looked at that yet...
Anyway, I found a bug, I think: In a newsgroup I replied to a topic as first person. Now there are 4 more replies after me but I'm still listed as the last poster.

george_proost
15 Jul 2001, 08:42
I have put in the new code, it is running.

I am struggling a bit with the [qx] definitions. Please help

newnews.pl leaves a dummy zombie job active each time it has run! Please help


Btw FF, there is a way to link the posts more accurately... I do not know the code but have seen the opportunity. Some posts seem to detach themselves from threads but are clearly part of an existing thread. ( I like to avoid pruning )

current linking
==========
Subject Thread 1
Re: Subject Thread 1 post 1
Re: Subject Thread 1 post 2

seems to be the general rule, however, multilingual news/mail clients can use a different prefix eg :

AW: Subject Thread 1 post x (I think these are handled - German) but seen some stray ones

Fw: Subject Thread 1 post z ( i've seen a couple sneak through here)

Antw: Re: Subject Thread 1 post y (Antw: = Reply in Dutch, I think you can strip that off as standard)

These are just comments, if you do see the opportunity and you feel it is relevant .. please.

I picked these examples from here (http://www.mainframeforum.com/forumdisplay.php?s=&forumid=1011) (currently reloading)

orca
15 Jul 2001, 12:27
George, I saw now where you have the problem. I had this first, too. You can fix it if you fill EVERY line of the bb-Code. Here's what I filled in for q1:

vB Code Tag: q1
vB Code Replacement: &nbsp;&nbsp;<font size="1" color="blue">{param}</font>
vB Code Example: Usenet1
vB Code Explanation: Used for Usenet
Use {Option}?: No.

Hope this helps.

fastforward
15 Jul 2001, 17:27
Originally posted by george_proost
I have put in the new code, it is running.

I am struggling a bit with the [qx] definitions. Please help

newnews.pl leaves a dummy zombie job active each time it has run! Please help


Btw FF, there is a way to link the posts more accurately... I do not know the code but have seen the opportunity. Some posts seem to detach themselves from threads but are clearly part of an existing thread. ( I like to avoid pruning )

current linking
==========
Subject Thread 1
Re: Subject Thread 1 post 1
Re: Subject Thread 1 post 2

seems to be the general rule, however, multilingual news/mail clients can use a different prefix eg :

AW: Subject Thread 1 post x (I think these are handled - German) but seen some stray ones

Fw: Subject Thread 1 post z ( i've seen a couple sneak through here)

Antw: Re: Subject Thread 1 post y (Antw: = Reply in Dutch, I think you can strip that off as standard)

These are just comments, if you do see the opportunity and you feel it is relevant .. please.

I picked these examples from here (http://www.mainframeforum.com/forumdisplay.php?s=&forumid=1011) (currently reloading)
The subject is not used in these examples. The messages are threaded according to the 'References' field in the article header. The examples you are showing are simply where people are not using their news clients correctly and are starting a new thread to answer an existing thread.

george_proost
15 Jul 2001, 20:14
thanks Paul, sorry for some of the stupid questions.

I got the quotes working forgot the ; use { option } no

looking gooood.

Now the zombies, anyone else with them. I now have 197 Zombies ??

v0n
17 Jul 2001, 12:50
I'm still trying to work out solution to posts being loaded with threadid 0 and bodies not showing up under threads ( I know it's boring). By now I tried three perl versions on two machines with same effect.
Yet I know it worked, I had it running till last rc of vbb2.0. Then on some point it just stopped. Even if i do a fresh install of vbb 2.0.1 and install snapshots from my data the usenet posts still get inserted with thread = 0. There must be some method to this madness.
Is there any possiblity that this bug is caused not by perl version or DBI DBD modules but by database structure? Maybe my base vbb is missing something, some index or row? Is there any tool to compare structure with original vbb setup?

captnroger
17 Jul 2001, 13:45
So what's the preferred news feed source for use with this hack?

fastforward
17 Jul 2001, 14:54
Originally posted by v0n
I'm still trying to work out solution to posts being loaded with threadid 0 and bodies not showing up under threads ( I know it's boring). By now I tried three perl versions on two machines with same effect.
Yet I know it worked, I had it running till last rc of vbb2.0. Then on some point it just stopped. Even if i do a fresh install of vbb 2.0.1 and install snapshots from my data the usenet posts still get inserted with thread = 0. There must be some method to this madness.
Is there any possiblity that this bug is caused not by perl version or DBI DBD modules but by database structure? Maybe my base vbb is missing something, some index or row? Is there any tool to compare structure with original vbb setup?
It's not the database structure. As you know the problem is that the mysql_insertid() function is not returning a threadid, postid or whatever id it asks for. The problem could be either in Perl, MySQL, or most likely, a combination of both due to a version mismatch.

The reason it seemed to suddenly 'stop' working is that previous versions of the script checked to see if the id was NULL and if it was, it would re-query the table to find it. This was only in place until we determined what the problem was with mysql_insertid. It added a lot of overhead and made the script almost unusable when the function wasn't working.

I have the latest version of MySQL, Perl 5.0053 and the latest version of the modules. This works.

Shaman
17 Jul 2001, 18:00
Mine was failing with:

---

Query failed:
INSERT LOW_PRIORITY INTO post (title,allowsmilie,threadid,dateline,nntpdateline,pagetext,visible,msgid,ord,use rname,userid,nntpfrom,ipaddress,isusenetpost) VALUES ('wanted',1,,995388228,945544816,'Need somebody to clean apartment.\n\ncgardner@home.com\n','1','<Q7R64.13866$TT4.520532@news1.rdc2.on.home.com>','0','C. Gardner',0,'&quot;C. Gardner&quot; &lt;cgardner@nospam.home.com&gt;',0,1)

DBD::mysql::db do failed: You have an error in your SQL syntax near '995388228,945544816,'Need somebody to clean apartment.\n\ncgardner@home.com\n','' at line 1 [for statement ``SELECT a.nntpposter, a.forum, a.msgid, a.dtm, a.subject, a.poster, a.email, a.body, a.ord, a.threadid, a.postid FROM usenet_article AS a LEFT OUTER JOIN post AS b on (a.msgid=b.msgid) WHERE a.forum = 26 AND b.msgid is null AND (refs = '' OR refs is NULL OR refs = '(none)') AND LEFT(subject,3) <> 'Re:' ORDER BY dtm'']) at ./newnews.pl line 594, <SOCK1> line 11165.

---

This was with DBI v1.16 and MySQL module v1.21xx

Note that threadid was nonexistant (which is the error in the query).

Now it is working perfectly with:

MySQL v 3.23.36
Perl 5.6.1
DBI v1.18
MySQL module from Msql-Mysqjl-Modules v1.2216

v0n
18 Jul 2001, 03:30
Originally posted by fastforward
I have the latest version of MySQL, Perl 5.0053 and the latest version of the modules. This works.

Separate installation of 5.0053 saved the day. Strangely enough when modules were installed from MCPAN -shell it behaved just like before. Reinstalled again with all modules (especially DBD-mysql-2.0902) compiled "by hand" and posts have threadid again.
I'm still having a bit of a problem with indexing (maybe due to posts indexed before, I will see tomorrow with indexing on fresh posts).

fastforward - let God reward you with Mercedes Benz for this mod. ;)

fastforward
18 Jul 2001, 05:39
Originally posted by v0n
fastforward - let God reward you with Mercedes Benz for this mod. ;)
Yeasss... A 500SL convertible would do nicely thank you :D


That reminds me of a great signature I saw on usenet:

There are a number of mechanical devices which increase sexual arousal, particularly in women. Chief among them is the Mercedes-Benz 380SL convertible.

george_proost
18 Jul 2001, 08:07
Yes, all is working.

Thanks all for your help. Yip, FF you deserve your Merc.
Now only my DNS woes!

orca
18 Jul 2001, 08:09
Originally posted by captnroger
So what's the preferred news feed source for use with this hack?

Well, that depends on you. Make a search on google or so for free newsserver.

andy_C
18 Jul 2001, 14:53
usenetserver.com seem good. Haven't had much experience with them, but they seem professional and cheap.

orca
18 Jul 2001, 15:33
I have missing posts in the thread (from Usenet). Is this also related to the version problem?

andy_C
18 Jul 2001, 15:36
I think that it is definitely possible. It's one of the errors that clears when you upgrade away from a free server.

andy_C
18 Jul 2001, 18:28
This hack can't handle groups with "+" signs in the usenet group name. Any idea how to circumvent this problem? I've tried escaping it with a \, but no luck. I guess I'd need to alter the perl script, although there may be a better way. any ideas?

fastforward
18 Jul 2001, 18:31
Originally posted by andy_C
This hack can't handle groups with "+" signs in the usenet group name. Any idea how to circumvent this problem? I've tried escaping it with a \, but no luck. I guess I'd need to alter the perl script, although there may be a better way. any ideas?
The trick is to add them manually via phpMyAdmin. I have a few groups with a + in them. It just means you can't edit them via the CP. It's on my 'todo' list... near the bottom :)

andy_C
18 Jul 2001, 18:33
Thank you so much fastforward. Although it's difficult to say thanks, I'd like to do it. You've provided a great many of us with a fantastic tool (esp. for marketing etc) and I really appreciate it. I'll check out the "+" thing and hopefully it should work as beautifully as the rest of the script does.

Dioxin
19 Jul 2001, 02:15
one short question:

what does this error message mean:

Processing headers for article 154365... OK
NNTPERROR: 421 No next to retrieve.
Processing headers for article 154366... OK
NNTPERROR: 421 No next to retrieve.
Processing headers for article 154367... OK
...

it appears not often, but occasionally :)

thanx in advance, dioxin :)

fastforward
19 Jul 2001, 02:29
Originally posted by Dioxin
one short question:

what does this error message mean:

Processing headers for article 154365... OK
NNTPERROR: 421 No next to retrieve.
Processing headers for article 154366... OK
NNTPERROR: 421 No next to retrieve.
Processing headers for article 154367... OK
...

it appears not often, but occasionally :)

thanx in advance, dioxin :)
Basically, it just means the server said it had an article, but it turned out it didn't really have it after all.

The server rarely reports how many articles it has accurately. Nothing to worry about, it doesn't affect the completion of messages or anything.

It actually happens quite often with my news provider. You must have a reliable one. :) There seems to be no obvious way to supress the warnings I'm afraid.

DarkReaper
19 Jul 2001, 03:39
I'm having problems with your usenet hack. After fixing all the problems with the SQL statements, when I modified newreply, it stopped working. It would still post the thread, and execute the phpinclude, but that was it. Here's the offending line:

$DB_site->query("INSERT INTO post (postid,threadid,title,username,userid,dateline,nntpdateline,attachmentid,pagete xt,allowsmilie,showsignature,ipaddress,iconid,visible,inreplyto) VALUES (NULL,'$threadid','".addslashes(htmlspecialchars($title))."','".addslashes(htmlspecialchars($postusername))."','$bbuserinfo[userid]','".time()."','".time()."','$attachmentid','".addslashes($message)."','$allowsmilie','$signature','$ipaddress','$iconid','$visible','$inreplyto')");


This could have to do with the above bug, but post counts aren't working either. When I post in a thread, my count stays the same.

dwh
19 Jul 2001, 03:42
I haven't installed the upgrade yet, plan to maybe tonight/tomorrow, but I just noticed that most of the groups are populating but one group is not populating even though using the same newsserver I know there are posts....dunno if the new version fixes this.,

DarkReaper
19 Jul 2001, 04:57
Ok, what the hell is going on here. Even after replacing my newthread and newreply files with the orignal ones from the vB zip, it still doesn't work! It brings up a blank page with only the phpinclude information loading, but when I hit refresh, it works fine?! How can I fix this, what is wrong with it??

orca
19 Jul 2001, 07:58
Just a quick question relating to binaries: If a newspost has a binary included, does the script remove the whole message or does it just remove the binary? For me, it seems that it removes the whole message.

Dioxin
19 Jul 2001, 11:17
thanx for your answer, fastforward :)

i have another one:

there are still many many threads awaiting for manual post insertion. is this normal? how can i avoid this?

what's the meaning of orphan? i'am a german guy and i don't understand this option in control panel. should i set it to 5 or a higher level.

thanx,

regards, dioxin :)

fastforward
19 Jul 2001, 17:57
Originally posted by Dioxin
thanx for your answer, fastforward :)

i have another one:

there are still many many threads awaiting for manual post insertion. is this normal? how can i avoid this?

what's the meaning of orphan? i'am a german guy and i don't understand this option in control panel. should i set it to 5 or a higher level.

thanx,

regards, dioxin :)
This question comes up time after time. Here are the reasons:

Posts are inserted into the forums if and only when these two conditions are true:

a post with a msgid that appears in the list of references in the header of the new article is already in the forum.
the ord column of the new article is greater than or equal to the ord column of the existing post in the forum.

There are various reasons why you may have orphan replies with no where to go:

the thread starter has already been expired from the news server and it was never retrieved.
the thread starter has already been expired from your forums based on your control panel settings.
the thread starter was rejected by your news server due to spam
the thread starter was rejected by your spam filters
the thread starter was rejected due to exceeding the maximum message length you have set.


The orphan setting is simply there to delete these orhan records from the usenet_article table. If the post to which the article replies has not appeared after 2 days, you can pretty much guarantee it's never going to arrive. The only reason it is configurable is to allow a full initial historical news pull that may go back several months.

DarkReaper
19 Jul 2001, 21:00
I still need help, here's the problems I had:

I got loads of errors when running those 2 SQL scripts.
Somewhere during the install, my whole boards got screwed. newthread.php was showing a blank screen except for the ad(which is in the phpinclude) and so was newreply, but they still posted threads/replies. However, when posting, it didn't show up as a new thread in the forum stats(threads, replies...), it didn't change the user's post count, and new replies didn't show up under the thread stats(replies, views..) but views did.

What am I doing wrong?? Or is it something wrong with the script?

fastforward
19 Jul 2001, 21:44
Originally posted by DarkReaper
I still need help, here's the problems I had:

I got loads of errors when running those 2 SQL scripts.
Somewhere during the install, my whole boards got screwed. newthread.php was showing a blank screen except for the ad(which is in the phpinclude) and so was newreply, but they still posted threads/replies. However, when posting, it didn't show up as a new thread in the forum stats(threads, replies...), it didn't change the user's post count, and new replies didn't show up under the thread stats(replies, views..) but views did.

What am I doing wrong?? Or is it something wrong with the script?
The script does nothing that affects your php pages. The errors can only be a result of your edits. I suggest reverting all your php files back to the originals and making each change individually. In fact, don't make any changes until you have the news coming in.

DarkReaper
19 Jul 2001, 22:07
I reverted the files back to the original ones, and still got the error. I'm assuming it has something to do with the database modifications. I got lots of errors when trying to install those, and corrected them, but something could have screwed up. Its really weird though. The error is there, but when I hit refresh it goes away.

fastforward
19 Jul 2001, 22:34
Originally posted by DarkReaper
I reverted the files back to the original ones, and still got the error. I'm assuming it has something to do with the database modifications. I got lots of errors when trying to install those, and corrected them, but something could have screwed up. Its really weird though. The error is there, but when I hit refresh it goes away.
The database changes shouldn't affect anything (at least not with those symptoms).

It could be you have cached pages somewhere (locally or a remote proxy). The cached pages are from when you were getting the errors. The refresh obviously forces a new page and the corrected file is retrieved.

v0n
20 Jul 2001, 20:33
There is a funny "bug" in the newest version, although, since I was going through all the different perl versions I'm not quite sure if it's not the perl modules that should be blamed; one thing is certain - previous versions of Usenet hack on my board didn't do it:

I've noticed (on my own example) that posts sent from news client (not the board) with nick names in square brackets (good example - my usenet nick is [MRK]) are displayed on the board either as Anonymous or using first part of e-mail before '@' sign. Any idea why?

fastforward
20 Jul 2001, 22:58
Originally posted by v0n
There is a funny "bug" in the newest version, although, since I was going through all the different perl versions I'm not quite sure if it's not the perl modules that should be blamed; one thing is certain - previous versions of Usenet hack on my board didn't do it:

I've noticed (on my own example) that posts sent from news client (not the board) with nick names in square brackets (good example - my usenet nick is [MRK]) are displayed on the board either as Anonymous or using first part of e-mail before '@' sign. Any idea why?
Email addresses are extracted from the 'from' line using a different module than the previous script versions. It now uses Mail::Address whereas it used to use Email::Find. It appears it simply ignores anything in the square brackets. I'll see what I can do about it.

fastforward
21 Jul 2001, 08:31
Release 20010721 fixes a few minor problems with newnews.pl.

Fixed the problem of angled brackets being shown as hashes for the 'from' line of usenet posts.
Fixed a problem with the article loader that may have caused duplicates if the message counters got reset.
Fixed the possibility of endless looping problem when re-importing local threads.
Simplified article batch pull routine.
Cleaned up console messages.
Put in a work-round for the Mail::Address problem with square brackets.
Fixed problem with signature going out even though the outgoing post is set to not show it.

To upgrade, you just need to replace your existing newnews.pl with the one in this package (http://britishexpats.com/download/usenet_gateway.tar.gz).

I'm off on vacation today (Saturday), so I won't be around for the next week. byeeee! :)

The latest version of this hack will always be in this first post of the thread.

BassWriters
23 Jul 2001, 19:19
I'm thinking of developing a site centering around a vBulletin and especially using this great Usenet hack.

My question is how big does the database get with all of the usenet posts? Are we talking up to a couple gigs, or into the tens and hundreds?

I know it's all variable, depending on the number of Usenet posts, I was just curious what to expect.

Thanks,
Cameron

JohnV
23 Jul 2001, 20:54
Greetings:


Well, I've been trying to figure this out for a week, so I guess I'll start begging for help here =)

I've got the newsgroups importing just fine, I have over 100,000 old usenet posts archived now, and new ones are filtering in every day. Everything in this department works great.

Where the problem comes, is outgoing. I have a dedicated news server here, and I can see that it's never making any attempt to send outgoing threads and replies. Looking in my database, the two outgoing fields are always empty, etc. It's simply not processing any posts that I make to the newsgroup forums as outgoing.

Any ideas? I have the latest version of everything required, running under apache with php as isapi, and it's running on a dedicated win 2000 box.

george_proost
25 Jul 2001, 14:26
The new code is performing wonderfully. Thankyou very much.

However, a small parsing/dellimeting error :

Query failed:
INSERT IGNORE INTO usenet_outgoing(poster,email,signature,newsgroup,subject,body,threadid,postid)
VALUES ('Mic Murph','xxx@xxx.com',,'ibmpub.java.os390','Access
to Environment Variables from JNI native code','A while back, I developed a Java & C/C++ native wrapper for our company\'s
data management product (tableBASE). We\'re running on
OS/390 MVS V2R8 using JDK 1.1.8 under USS. \r\n\r\nThe Java
wrapper statically loads the C/C++ class when it is instantiated.
\r\n\r\nIn the C/C++ native routine, I used a "fetch()" library call
to locate the function pointer to a HLASM routine, which is stored
in a pre-existing MVS namespace in a PDS(not PDSE) link-editted load library. \r\n\r\nI used the "STEPLIB" environment variable in my BASH script to refer to the two load libraries that had the
HLASM stub and the additional overlays that it calls. \r\n\r\nNote that this results in a HLASM call wrapped in a C/C++ JNI native
method wrapped in a Java class. We used this approach since
the HLASM interface is well known by our customer base. \r\n\r\nI set up an RMI-distribution for the Java wrapper using a BASH
script to set/export various environment variables. We\'ve been
experimenting with different client architectures, such as
servlet/JSP, standalone application over RMI etc. \r\n\r\nThe
setup works and is pretty speedy, considering we\'re still using
the May 2000 edition of the 1.1.8 JDK. \r\n\r\nNow, we\'re
starting to scale up for productionalize by running the JNI
wrapper from within a servlet in WebSphere 3.02. And when we
run the servlet, the Java wrapper class is able to load the C/C++
native wrapper OK. \r\n\r\nThe problem is that the C
program\'s "fetch" of our HLASM load library fails, as if it\'s not finding our STEPLIB environment variable. \r\n\r\nMy understanding is that WAS inherits any environment variable
settings from the HTTP 5.2 server config (in /etc/httpd.envvars). \r\nThis is where we\'ve put both our LIBPATH entry (for our
native .SO DLL which is being found by the loader) and our
STEPLIB variable (for our HLASM load library which apparently
ISN\'T being found). \r\n\r\nI\'d prefer not to have to use
the "setenv()" library calls to set/get the environment variables
from my C/C++ native routine.\r\n\r\nAny suggestions? \r\n\r\nThanks, Michael Murphy \r\n\r\nP.S. This is actually a cross-
post from the new forum/portal "www.mainframeforum.com".
Check it out!',30239,83853)


DBD::mysql::db do failed: You have an error in your SQL syntax
near ''ibmpub.java.os390','Access to Environment Variables from
JNI native code','A wh' at line 1 at www/admin/newnews.pl line
596, <SOCK3> line 2.

:)

welcome back

:D

webhost
27 Jul 2001, 02:33
I really need some help here please

Need some help everything was working great than all of sudden my usenet posts where being added to my board post's . I had but the option in to avoid this but for some reason it quit working, when I purge usenet groups the board and the thread count along with usenet counts decrease. This is how I have it in forums/index.php file, also why would purging timeout before finishing? here;s what I have Found it had to remove lines in bold below.

// get total posts
$countposts=$DB_site->query_first('SELECT COUNT(*) AS posts FROM post');
$totalposts=$countposts['posts'];
if ($totalposts=='') {
$totalposts=0;
}

// START USENET HACK //
$countthreads=$DB_site->query_first('SELECT COUNT(*) AS threads FROM thread WHERE isusenetpost=0');
// END USENET HACK //

$countthreads=$DB_site->query_first('SELECT COUNT(*) AS threads FROM thread');
$totalthreads=$countthreads['threads'];
if ($totalthreads=='') {
$totalthreads=0;
}

// START USENET HACK //
$countusenetposts=$DB_site->query_first('SELECT COUNT(*) AS posts FROM post WHERE isusenetpost=1');
$totalusenetposts=$countusenetposts['posts'];
if ($totalusenetposts=='') {
$totalusenetposts=0;}
$countusenetthreads=$DB_site->query_first('SELECT COUNT(*) AS threads FROM thread WHERE isusenetpost=1');
$totalusenetthreads=$countusenetthreads['threads'];
if ($totalusenetthreads=='') {
$totalusenetthreads=0;}
// END USENET HACK
// Total Thread Views

// get newest member

webhost
27 Jul 2001, 03:18
I will clean this up and show what I have done and where I am at.

First I still have first hack installed, Haven't upgraded yet.

Hack had been working fine until today.

I had earlier set up the option where the usenet posts would not count against the board total all had been going well for the last couple of months. Today I look and the board post count had the usenet count added to it also the thread count. I looked over script placement and it was correct according to the instructions. I went to my database and sure enough all board post and usenet posts showed up in the post table all 23,000 of them. So I went to the control panel and first I purged all of the newsgroups; post's came down when I did this both for board post and also for usenet post's. After this was complete I updated all counters. I still had some newsgroups showing contents even though I had just purged all of them. So I went to that forum and pruned them and them updated counters. I went back to the usenet section of the control panel and all newsgroups showed zero. I went back to board and even though I had just zeroed out everything in the usenet control panel; On the board it was still showing 181 posts and the board total was high by that number 181. I went back to the database post table and sure enough there were 181 usergroup post's still there. I then went and ran a test on 3 newsgroups and reactivated them, Information came in and same thing they are adding to my main board total.
First any ideas what is doing this?
Second what is the easiest way to remove those 181 post's in the post table?

Any help would be Welcomed. Have working again just need to know why the posts would not delete when the rest did and how to accomplish removing them?

Thanks Joey:confused:

James Cridland
27 Jul 2001, 22:08
This hack looks just incredible, and I'm excited about getting it to work.

I'm having problems installing the perl modules, reporting an error...Checking if your kit is complete...
Looks good
Writing Makefile for Text::Autoformat
-- OK
Running make test
'test' is not recognized as an internal or external command,
operable program or batch file.
test -- NOT OK
Running make install
make test had returned bad status, won't install without force

I'm using Windows 2000 running ActivePerl - should I stop now, or should I persevere?

george_proost
28 Jul 2001, 14:11
I run on linux ... the hack is really great.

Keep on going at it till you get it right.
USENET-Gateway : is the most important hack for me.

Second hack is the overgrow archive hack... although I have my reservations about the effectiveness. I have made some changes which I'll release when I'm done, it'll be called <overgrows positively enhanced referral attractor> still needs a bit of final display work.

archive.mainframeforum.com

Some enhancements:
-Displays only levels with content allowing spiders to navigate faster, cleaner and always with something to digest.
-Employs text sizing, highlighting etc.. to attract spiders. They see the important stuff right away.

fastforward
28 Jul 2001, 22:35
Originally posted by george_proost
The new code is performing wonderfully. Thankyou very much.

However, a small parsing/dellimeting error :

Query failed:
INSERT IGNORE INTO usenet_outgoing(poster,email,signature,newsgroup,subject,body,threadid,postid)
VALUES ('Mic Murph','xxx@xxx.com',,'ibmpub.java.os390','Access
to Environment Variables from JNI native code','A while back, I developed a Java & C/C++ native wrapper for our company\'s
data management product (tableBASE). We\'re running on
OS/390 MVS V2R8 using JDK 1.1.8 under USS. \r\n\r\nThe Java
wrapper statically loads the C/C++ class when it is instantiated.
\r\n\r\nIn the C/C++ native routine, I used a "fetch()" library call
to locate the function pointer to a HLASM routine, which is stored
in a pre-existing MVS namespace in a PDS(not PDSE) link-editted load library. \r\n\r\nI used the "STEPLIB" environment variable in my BASH script to refer to the two load libraries that had the
HLASM stub and the additional overlays that it calls. \r\n\r\nNote that this results in a HLASM call wrapped in a C/C++ JNI native
method wrapped in a Java class. We used this approach since
the HLASM interface is well known by our customer base. \r\n\r\nI set up an RMI-distribution for the Java wrapper using a BASH
script to set/export various environment variables. We\'ve been
experimenting with different client architectures, such as
servlet/JSP, standalone application over RMI etc. \r\n\r\nThe
setup works and is pretty speedy, considering we\'re still using
the May 2000 edition of the 1.1.8 JDK. \r\n\r\nNow, we\'re
starting to scale up for productionalize by running the JNI
wrapper from within a servlet in WebSphere 3.02. And when we
run the servlet, the Java wrapper class is able to load the C/C++
native wrapper OK. \r\n\r\nThe problem is that the C
program\'s "fetch" of our HLASM load library fails, as if it\'s not finding our STEPLIB environment variable. \r\n\r\nMy understanding is that WAS inherits any environment variable
settings from the HTTP 5.2 server config (in /etc/httpd.envvars). \r\nThis is where we\'ve put both our LIBPATH entry (for our
native .SO DLL which is being found by the loader) and our
STEPLIB variable (for our HLASM load library which apparently
ISN\'T being found). \r\n\r\nI\'d prefer not to have to use
the "setenv()" library calls to set/get the environment variables
from my C/C++ native routine.\r\n\r\nAny suggestions? \r\n\r\nThanks, Michael Murphy \r\n\r\nP.S. This is actually a cross-
post from the new forum/portal "www.mainframeforum.com".
Check it out!',30239,83853)


DBD::mysql::db do failed: You have an error in your SQL syntax
near ''ibmpub.java.os390','Access to Environment Variables from
JNI native code','A wh' at line 1 at www/admin/newnews.pl line
596, <SOCK3> line 2.

:)

welcome back

:D
I've just encountered this error myself. It happens when a post is made that has the 'show signature' option unchecked. I'll post a fix later tonight.

JenniferS
29 Jul 2001, 03:13
to say thank you so very much, fastforward. this hack is extremely valuable, and I am thrilled to have it working on my board.

i look forward to your future contributions.

best wishes,
J

dwh
29 Jul 2001, 04:44
Originally posted by JenniferS
to say thank you so very much, fastforward. this hack is extremely valuable, and I am thrilled to have it working on my board.

i look forward to your future contributions.

best wishes,
J

Jennifer, I checked out your site, looks interesting tho' I'm still not clear on the main topic or purpose of the site...but I got I guess it was probably javascript error on the first thing I clicked.

You also may like to know that putting the cursor over your ip icon displays <!--Template IP code etc.. --> probably bad alt tag or else a missing quote somewhere..

JenniferS
29 Jul 2001, 05:36
i still have the system spitting out template identification comments which may or may not have been responsible for your javascript dead end.

which is to say, sorry you couldn't get in, and, my purpose is: knowledge accumulation.

dwh
29 Jul 2001, 06:42
I don't think it was javascript as I looked back. I think you have a quotes problem. Either you didn't close a quote, or maybe you have some code in phpinclude and mistakenly had something like

$variable="<font "Times New Roman">Words</font>" and that messed up the code....if that's the case, to fix it try $variable="<font 'Times New Roman'>Words</font>" instead.

fastforward
29 Jul 2001, 06:47
Release 20010729 fixes two problems with newnews.pl.

Fixed SQL error when post is made with signature set to not be shown.
Fixed posting time for local posts made in usenet forum. The nntpdateline was being shown as 1969.

To upgrade, you just need to replace your existing newnews.pl with the one in this package (http://britishexpats.com/download/usenet_gateway.tar.gz).


The latest version of this hack will always be in this first post of the thread.

James Cridland
29 Jul 2001, 13:07
Installing this on an ActivePerl build for Windows 2000.

So far, I've managed to get further installing the modules. ActivePerl uses PPM modules. To install, type "PPM" into a command box, then type "install [package name]".

Which is where I've managed pretty well, until the last three. Package names differ slightly, but this is what I've installed so far, with their CPAN equivalent names...


cpan> install News::NNTPClient
News-NNTPClient

cpan> install Net::SMTP
NetServer-SMTP

cpan> install DBD::mysql::Install
DBD-Mysql

cpan> install Text::Autoformat
Text-Autoformat

cpan> install Date::Parse
Date-Manip

cpan> install MIME::Tools
MIME-tools

cpan> install URI::Find
URI-Find

cpan> install MIME::WordDecoder
?

cpan> install HTML::Entities
?

cpan> install Mail::Address
?

...and as you can see, I'm stuck with the last three. I'm still persevering, and I hope this is useful to some who are trying to install it on my kind of setup (ActivePerl, Apache, Win2000).


Until then, I've found a script which offers both read and write functionality just through PHP - check out http://forums.mediauk.com/usenet/ for a demonstration and a link to the source. However, it doesn't use mySQL or indeed any database structure (probably quite easy to patch for a programmer who knows what they're doing, mind.)

It would make more sense, to this user at least, to have a fully native PHP solution rather than having to use Perl: since, as I understand it, all the necessary code's already in PHP.

webhost
29 Jul 2001, 15:06
I am currently using this version 20010528,

What do I need to do to upgrade just upload the newnews.pl or more?

fastforward
29 Jul 2001, 18:00
Originally posted by webhost
I am currently using this version 20010528,

What do I need to do to upgrade just upload the newnews.pl or more?
There's other changes since that version. There's also aditional modules that are needed. The best bet is to download the interim packages upgrade in order.
usenet_gateway_v20010714__201.tar.gz (http://britishexpats.com/download/usenet_gateway_v20010714__201.tar.gz)
usenet_gateway_v20010721__201.tar.gz (http://britishexpats.com/download/usenet_gateway_v20010721__201.tar.gz)
Then finally, do the latest upgrade from this thread.

fastforward
29 Jul 2001, 18:47
Originally posted by James Cridland
cpan> install MIME::WordDecoder
?
cpan> install HTML::Entities
?
cpan> install Mail::Address
?

...and as you can see, I'm stuck with the last three. I'm still persevering, and I hope this is useful to some who are trying to install it on my kind of setup (ActivePerl, Apache, Win2000).

Activestate provide ports for all of these
WordDecoder is part of the MIME-Tools package.
HTML::Entities is part of the HTML-Parser package
Mail::Address is part of the MailTools package

I recently changed some of the modules that were being used to ensure that there were ActiveState modules available.

On win32 systems, you will have to comment out the bit that checks for a previous running version near the top of newnews.pl.

It would make more sense, to this user at least, to have a fully native PHP solution rather than having to use Perl: since, as I understand it, all the necessary code's already in PHP.
There are no native NNTP functions built into php. There is IMAP support (assuming you compiled it with the appropriate options), but that doesn't provide you with full newsreader capabilities. That aside, the reading of the news is actually the easy bit. The main guts of the script deals with news synchonisation & parsing and database access and vB integration. There is no doubt you could write it in PHP, but you wouldn't have the Perl modules to help you. You would also have to install the binary version of php to allow periodic execution of the script.

James Cridland
29 Jul 2001, 20:32
I'm nearly there! :)
I've made some additions to the installation for other W2K/ActivePerl users, which (once everything works) I'll post here.

My guess at "Date-Manip" in PPM was the wrong package to install. Turns out the right one is "TimeDate", and not Date-Manip.

fastforward
29 Jul 2001, 21:18
Originally posted by James Cridland
I'm nearly there! :)
I've made some additions to the installation for other W2K/ActivePerl users, which (once everything works) I'll post here.

My guess at "Date-Manip" in PPM was the wrong package to install. Turns out the right one is "TimeDate", and not Date-Manip.
Glad you're getting there. If you don't mind I'll include your ActiveState instructions with the package once you've got it all going. I'll also see if there's any other way of checking for an existing instance without using grep that will work on both Unix and Win32. In the meantime, you'll just have to comment it out.

James Cridland
29 Jul 2001, 22:28
Here's the additional stuff, which you can include in any future releases. The numbers at the beginning are the original installation notes. I hope it's useful.

Initial thoughts are, certainly when using news.btinternet.com (my local ISP), that newnews.pl is very slow in grabbing new messages - typically up to 90 seconds per message. For this reason, I'm currently setting it to run once a day. Is this about right?

It strikes me that running a local news server would speed this process up; is there a suitable freeware news server available somewhere?

This is a brilliant addition; now in use (though not extensively) at http://forums.mediauk.com/


1. [Installing the modules]
WIN32/ActivePerl: To install these, type "PPM" into a command box. Then, type:
install News-NNTPClient
install NetServer-SMTP
install DBD-Mysql
install Text-Autoformat
install TimeDate
install MIME-tools
install URI-Find

3. [Running the sql scripts]
WIN32: To run these using mysql: type "mysql" at the command prompt. Then, type:
connect vbfor [where vbfor is the name of your database]
source vB_sql_structure.sql
source vB_sql_data.sql

8. [Editing newnews.pl]
WIN32: Comment out line 53 - line 67.

Running:
WIN32: Set up a shortcut (either on your desktop or in your start menu) to point to: "perl c:\path\newnews.pl"
Then, use Scheduler (Start|Programs|Accessories|System Tools|Scheduled Tasks) to run this program at specified times.

fastforward
29 Jul 2001, 23:06
Originally posted by James Cridland
Initial thoughts are, certainly when using news.btinternet.com (my local ISP), that newnews.pl is very slow in grabbing new messages - typically up to 90 seconds per message. For this reason, I'm currently setting it to run once a day. Is this about right?
90 seconds is most definately NOT about right. It shouldn't even take that long for a full batch of 200 or 300. try putting some debugging lines in the code to see exactly where it goes wrong.

James Cridland
30 Jul 2001, 00:03
Well, I might need your help here... :)

I'll have a play tomorrow.

webhost
30 Jul 2001, 15:31
ok Paul got first upgrade done, this oneusenet_gateway_v20010714__201.tar.gz

But I forgot a step it said to write down the useragent setting and then put back in, well I forgot, now that I look for it can not find anyway, first where is it and second what can I do now since I forgot it?

Joey


Update fixed

fastforward
30 Jul 2001, 17:40
Originally posted by webhost
ok Paul got first upgrade done, this oneusenet_gateway_v20010714__201.tar.gz

But I forgot a step it said to write down the useragent setting and then put back in, well I forgot, now that I look for it can not find anyway, first where is it and second what can I do now since I forgot it?
Update fixed
Not a big problem. The useragent is used to identify posts on usenet that originated from your forums. The main use is to grab the NNTP server assigned message id for local posts and update the post in the forum.

If no posts were sent to usenet during your very last execution of newnews.pl, you will have no problems. If there was a post sent, then that post will never have it's message id updated and will not get any replies.

So, your best bet is to just pick another useragent (usually your site name) and forget about it. :)

webhost
30 Jul 2001, 19:01
ok all upgrades done. I am now current.
One question though I need some help with. Never messed with the bbcode's, How do I install those for replacement's q1,q2,q3, I made replacement variables but I know that probably is wrong because they are not working. Can someone direct me a little on this?

James Cridland
30 Jul 2001, 19:39
Yes, this foxed me for a while.

In your control panel, choose "Replacement Variables" and "add" - then fill in below as follows...

Replacement set: [your normal style]
Code to find:
Code to insert: &nbsp;&nbsp;<font size="1" color="blue">

Replacement set: [your normal style]
Code to find:
Code to insert: </font>

...and similar for the other two codes.

At least, that's how I did it.

webhost
30 Jul 2001, 19:49
if you look at image below this what I did for each, I tried
and as you can see also{q1}, which one is right and also the instructions didn't say anything about using a closing bracket like or {/q1] so which one is right and do I also have to make 3 more variables for closing, 1 for each q1,q2,q3

got it working

Joey

James Cridland
30 Jul 2001, 21:22
Originally posted by fastforward

90 seconds is most definately NOT about right. It shouldn't even take that long for a full batch of 200 or 300. try putting some debugging lines in the code to see exactly where it goes wrong.

Well, after a lot of fiddling...

To make a huge increase in the speed, comment out (with the closing brackets, of course):
"if ($h = $c->head($mid)) {" (in get_batch)
"if($b = $c->body($mid)) {" (just after the check for spam)

Then it speeds along, as you say it should (though it doesn't find any to import at the end...)

So, for this novice perl programmer - presumably "head" and "body" are in a module somewhere, and this module needs reinstalling?

tamarian
30 Jul 2001, 21:43
Thanks for the update Paul! I'm a slow upgrader, but just installed it after skipping several upgrades :)

Hope you enjoy your vacation!

tamarian
30 Jul 2001, 21:57
Originally posted by webhost
[B]if you look at image below this what I did for each, I tried
and as you can see also{q1}, which one is right and also the instructions didn't say anything about using a closing bracket like or {/q1] so which one is right and do I also have to make 3 more variables for closing, 1 for each q1,q2,q3


I don't think you need the closing brackets. It worked fine for me just as in the instructions.

tamarian
30 Jul 2001, 22:00
Originally posted by webhost
ok all upgrades done. I am now current.
One question though I need some help with. Never messed with the bbcode's, How do I install those for replacement's q1,q2,q3, I made replacement variables but I know that probably is wrong because they are not working. Can someone direct me a little on this?

Oh, I think I know your problem, since you mentioned you never added bbcodes before.

Make sure you enter the example part

delete the [ / q ] , and just add them as in the install doc.

in the example just put test

Could that be it?

Shaman
30 Jul 2001, 22:41
Bug in the subject line:

If there is a "$" in the subject of a new thread (as in "Pentium III 733, $750!", the newnews script will fail.

fastforward
30 Jul 2001, 23:42
Originally posted by tamarian


Oh, I think I know your problem, since you mentioned you never added bbcodes before.

Make sure you enter the example part

delete the [ / q ] , and just add them as in the install doc.

in the example just put test

Could that be it?
Another thing that's a bit confusing is the 'Use {option} ?' option. It defaults to yes, but you need to set it to NO

fastforward
30 Jul 2001, 23:46
Originally posted by James Cridland


Well, after a lot of fiddling...

To make a huge increase in the speed, comment out (with the closing brackets, of course):
"if ($h = $c->head($mid)) {" (in get_batch)
"if($b = $c->body($mid)) {" (just after the check for spam)

Then it speeds along, as you say it should (though it doesn't find any to import at the end...)

So, for this novice perl programmer - presumably "head" and "body" are in a module somewhere, and this module needs reinstalling?
hehe. I'm not surprised that speeds things up. Those two lines are the ones that actually grab the article (first the header, then the body) from the news server. :D

However, by doing that, you have established the problem is probably with the news server and not the script. Do you have access to another news server you can try? If not, PM me your IP and I will give you temporary access to my DNews server for testing.

fastforward
31 Jul 2001, 01:11
Originally posted by Shaman
Bug in the subject line:

If there is a "$" in the subject of a new thread (as in "Pentium III 733, $750!", the newnews script will fail.
Can you remember on what line the script failed?

Shaman
31 Jul 2001, 16:22
First things first: any idea when a version will be available for vB 2.0.3? I'm curious mostly because of the security fix, otherwise I'd be happy to leave it be.

Any chance you'd work a deal to include your code with their codebase? :D

There may be more to the problem, I couldn't immediately reproduce the issue. Will get back to you, but I know I didn't imagine it.

Sinema
31 Jul 2001, 17:22
Howdy

I was curious where I can get the latest version of usenet that has the fixes included in it?

So far I've started @ VBulletin.org and downloaded Usenet there and then went to view the thread and it was posted that I did not have the lastest version .. so I followed the thread here.

After viewing 7 pages of posts I've downloaded Usenet 3-4 times because each time I download it I find a fix or another new version ..

was curious if there was one post that had the fixes and the latest version in one central area that is updated as fixes or new versions come available?

Where it says Usenet I meant the Gateway Hack :-) not the service that provides newsgroups

george_proost
31 Jul 2001, 20:47
hiya Paul,

a little one,

in the signatures when a url in the format :

<url="http://www.xxx.com/yy.htm">linkname</url>

parses (incorrectly) and posts to USENET
it includes the urlname concatenated to the linkname like

www.xxx.com/yy.htmlinkname

instead of

linkname

hope this helps ...

george_proost
31 Jul 2001, 22:07
The post at the top always contatins the latest version + fixes

hope this helps

:)

Sinema
31 Jul 2001, 22:28
Yeah that's what I was thinking but when you see all these download links every few posts .. it can kinda throw you off lol


Thanks again!!
..

george_proost
31 Jul 2001, 23:09
:)

Paul, I use multiple news feeds, some fast, some slow, some free, some subscribed.

Is it possible to (in a very future release)

sort by newsfeeder (i.e.one login per newnews execution)
spawn a separate task out per newsfeeder group (wishfull thinking)
on exit of newnews (successful exit !!) start a cron job for a random start between a period range start the next x and x+y minutes (or something )
x = I need an update at least evey x minutes average
y = start no sooner than y minutes from last execution
so
next start time = now + y + random(fromset[x])

super wishful thinking

Sinema
31 Jul 2001, 23:34
Howdy again,

I've viewed the old thread .. and after 30+ pages I gave up .. it's like I find someone posting about this and the post above and below never mention how

okay here is the question

I've never used telnet for anything .. so how do I execute the newnews.pl via telnet ?

I downloaded a Telnet program from download.com .. just no clue how to execute the .pl ..

okay this is edited now :-)

I searched and found this

wget http://domain.to/your/newnews.pl --output-file=/dev/null

However when I type it and put the URL to my file nothing happens.. so I'm thinking that's not it ..

Any help would be great ... maybe this will help some other non-telnet users out ..

george_proost
01 Aug 2001, 06:15
goto the directory where newnews.pl is installed then type

perl newnews.pl

I advise you to be familliar with what you are doing. This change to vBulletin is NOT for the faint-hearted.

James Cridland
01 Aug 2001, 23:20
Originally posted by george_proost
I advise you to be familliar with what you are doing. This change to vBulletin is NOT for the faint-hearted.

I would agree with that. For this user, at least, the extra perl overhead has sadly meant that my little server can't cope with running it at the moment.

I think I'll try again later at some stage, but perl is now vaped (again!) from my server.

JenniferS
02 Aug 2001, 02:42
Sinema, I needed help with this hack. I even know quite a bit about Telnet. But I don't know how to install Perl modules or "read" the newnews.pl script. You may want to hire a professional Perl person to assist you getting set up. It wouldn't take them more than about two hours.

You also need a paid NNTP feed.

J

fastforward
02 Aug 2001, 04:14
Originally posted by george_proost
Is it possible to (in a very future release)

sort by newsfeeder (i.e.one login per newnews execution)

Not sure I understand. Can you elaborate?

spawn a separate task out per newsfeeder group (wishfull thinking)

Possible, and with Activestate now supporting 'fork', it should work under NT aswell. The problem is the load. On large batch jobs, the many successive MySQL inserts already causes a high server load. Imagine the problem with 3 or copies running simultaneously.

on exit of newnews (successful exit !!) start a cron job for a random start between a period range start the next x and x+y minutes (or something )

x = I need an update at least evey x minutes average
y = start no sooner than y minutes from last execution
so
next start time = now + y + random(fromset[x])

Yep... good idea. I'd probably make use of this so I'll look at adding it sooner rather than later :)

dwh
02 Aug 2001, 06:30
Just to clarify, I read correctly that this isn't 2.03 compliant yet? I have the old release installed but I want the new fixes but will wait if the new version is still in dev.

george_proost
02 Aug 2001, 11:05
Hi Paul, develop we must !

sort by newsfeeder (i.e.one login per newnews execution)
currenty my newnews does

FEED1 then FEED2 then FEED3 again FEED1 again FEED3

this is due to the order of processing (order of creating the feeds), it would be 'more efficient' if the feeds were sucked out one at a time like :

FEED1 then FEED2 then FEED3 and done.

spawn a separate task out per newsfeeder group (wishfull thinking) need power then. nice 'option' though,
however the question arose due to the FEED1 above being sloooow sometimes causing the faster feeds to timeout (killed by next suck session) resulting in degraded updates on the faster FEEDS

A solution would be to 'prioritise' each feed for sucking.

the CRON will solve some of the slow sucking as each session will run to completion, however I feel now an on/off toggle would be needed as well in the options for this.

---------------------------------------------------------------------------------
In short a prioritise on FEED not sort would address most of the requirements above.
---------------------------------------------------------------------------------

lol ... has become a requirement

so the result in may case would be:

FEED2 then FEED3 then FEED1 (perhaps all spawned and rescheduled ;) )

----------------------------------------------------------------------------------
Thanks for your enthusiasm to grow your tool.

(sounds like one of those enlarge your ... spams)


:D

fastforward
02 Aug 2001, 17:23
Originally posted by dwh
Just to clarify, I read correctly that this isn't 2.03 compliant yet? I have the old release installed but I want the new fixes but will wait if the new version is still in dev.
You'll probably find it works fine. Obviously the line numbers for the php edits may have shifted slightly, but just search for the relevant bits. Also, remember that none of the code changes are necessary for this hack to run (discounting the minor html addition for control panel access).

If necessary, I'll release updated instructions for 2.03 over the weekend.

dwh
02 Aug 2001, 18:19
When I'm done updating the templates, I'll try to upgrade to the latest version and let u know if there were any problems. In fact, I'll enjoy it this time because last time I just wanted it to work. This time I will try to understand better what you're doing.

orca
03 Aug 2001, 07:37
showthread.php has changed the most. Else the hack works fine with 2.0.3. For the stars hack, I had to edit admin/functions.php instead of showthread.php. I try this now also with this hack.

I just edited functions.php and it works. Instead of editing showthread.php you need to look for the same strings in admin/functions.php and add the code there.

shri
03 Aug 2001, 10:10
Just a quick FYI. I upgraded to 2.0.3 forgetting that newnews.pl was running as a cron job. So far no problems other than the fact that its missing from the control panels :)

Pie'oh'pah
03 Aug 2001, 12:30
Hi Paul,

first of all: Great hack, greatly appreaciate your efforts!

It would seem, i got 2 problems nobody else encountered so far.

When submitting the Options form in admin cp the following error appears all the time:

--------------------------------
Warning: GLOBAL variable declaration meaningless in main() scope in /homepages/26/d17680768/htdocs/ua/vbulletin/admin/usenet.php3 on line 353
Your changes have been saved.
Parse error: parse error in /homepages/26/d17680768/htdocs/ua/vbulletin/admin/usenet.php3 on line 378
--------------------------------

and when submitting the Groups:

---------------------------
Your changes have been saved.
Parse error: parse error in /homepages/26/d17680768/htdocs/ua/vbulletin/admin/usenet.php3 on line 378
----------------------------

I can connect to the newsserver anyways, but after several 430's im getting this:

-----------------------------
Query failed:
UPDATE LOW_PRIORITY forum SET replycount=replycount+1, threadcount=threadcount+1 ,lastpost=995470170,lastposter='Trubnik.Com',lastactivethread=' !!! TRUBNIK.COM ...' WHERE forumid IN ()

DBD::mysql::db do failed: You have an error in your SQL syntax near ')' at line 1 at ./newnews.pl line 597, <SOCK1> line 4600.
bash-2.04$
-------------------------------


Can you make anything out of it?

Thanx

-Alex

george_proost
03 Aug 2001, 12:44
Pie'oh'pah which release of PHP are you running? I had the same errors with PHP3, then I converted to PHP4 and no problems any more.

The errors (top2) can be ignored I think.

The other SQL error I do not know about.

The USENET code is acutally quite clean and runs without problems at may sites. If the installatuon instructions were followed to the T. It should all work ok.

But please try to upgrade to PHP4, this will really make an error report valid. I'm sure Paul will get to it soon... hang in there buddy

Pie'oh'pah
03 Aug 2001, 13:00
its php 4.0.4

fastforward
03 Aug 2001, 14:50
Originally posted by Pie'oh'pah

--------------------------------
Warning: GLOBAL variable declaration meaningless in main() scope in /homepages/26/d17680768/htdocs/ua/vbulletin/admin/usenet.php3 on line 353
Your changes have been saved.
Parse error: parse error in /homepages/26/d17680768/htdocs/ua/vbulletin/admin/usenet.php3 on line 378
--------------------------------

and when submitting the Groups:

---------------------------
Your changes have been saved.
Parse error: parse error in /homepages/26/d17680768/htdocs/ua/vbulletin/admin/usenet.php3 on line 378
----------------------------

I can connect to the newsserver anyways, but after several 430's im getting this:

-----------------------------
Query failed:
UPDATE LOW_PRIORITY forum SET replycount=replycount+1, threadcount=threadcount+1 ,lastpost=995470170,lastposter='Trubnik.Com',lastactivethread=' !!! TRUBNIK.COM ...' WHERE forumid IN ()

DBD::mysql::db do failed: You have an error in your SQL syntax near ')' at line 1 at ./newnews.pl line 597, <SOCK1> line 4600.
bash-2.04$
-------------------------------

Are you sure you're on php 4.04? That usenet.php error is caused by the 'foreach()' loop. The foreach construct is not supported in php3.

The second error is probably a direct result of this as it would have caused the usenet groups not to be mapped correctly to the local forums. This is why you notice there is nothing in the SQL clause in the 'IN ()' clause. It should be full of the forums that this message belongs to.

webhost
03 Aug 2001, 14:55
Paul he is on the same server as I am, Mind is working perfectly.

Joey php 4.0.5

webhost
04 Aug 2001, 00:44
iT is working but sometimes I get these errors when script runs

Any ideas? These were 2 different script runs, It still gets all messages and all newsgroups come back with a ok at the end of the run
============================================
Unmatched '<>' at /usr/local/lib/perl5/site_perl/5.6.1/Mail/Address.pm line 189, <SOCK1> line 177.
============================================
NNTPERROR: 421 No Next Article
NNTPERROR: 421 No Next Article
NNTPERROR: 423 No Such Article In Group
NNTPERROR: 421 No Next Article
NNTPERROR: 423 No Such Article In Group
NNTPERROR: 421 No Next Article
NNTPERROR: 423 No Such Article In Group
NNTPERROR: 421 No Next Article
NNTPERROR: 423 No Such Article In Group
NNTPERROR: 421 No Next Article
NNTPERROR: 423 No Such Article In Group
NNTPERROR: 421 No Next Article
NNTPERROR: 423 No Such Article In Group
NNTPERROR: 421 No Next Article
NNTPERROR: 421 No Next Article

fastforward
04 Aug 2001, 00:54
Originally posted by webhost
iT is working but sometimes I get these errors when script runs

Any ideas? These were 2 different script runs, It still gets all messages and all newsgroups come back with a ok at the end of the run
============================================
Unmatched '<>' at /usr/local/lib/perl5/site_perl/5.6.1/Mail/Address.pm line 189, <SOCK1> line 177.
============================================
NNTPERROR: 421 No Next Article
NNTPERROR: 421 No Next Article
NNTPERROR: 423 No Such Article In Group
NNTPERROR: 421 No Next Article
NNTPERROR: 423 No Such Article In Group
NNTPERROR: 421 No Next Article
NNTPERROR: 423 No Such Article In Group
NNTPERROR: 421 No Next Article
NNTPERROR: 423 No Such Article In Group
NNTPERROR: 421 No Next Article
NNTPERROR: 423 No Such Article In Group
NNTPERROR: 421 No Next Article
NNTPERROR: 423 No Such Article In Group
NNTPERROR: 421 No Next Article
NNTPERROR: 421 No Next Article
That isn't a true error. It's just telling you the article number you requested is not available. Your lastmsgid may have got out of sync. this can happen if your news server rebuilds their index or expires a lot of news in one go. Check the latest article number using a convential news reader.

tamarian
04 Aug 2001, 01:47
Originally posted by fastforward

You'll probably find it works fine. Obviously the line numbers for the php edits may have shifted slightly, but just search for the relevant bits. Also, remember that none of the code changes are necessary for this hack to run (discounting the minor html addition for control panel access).

If necessary, I'll release updated instructions for 2.03 over the weekend.

Paul, did you (or any usenet hack user) successfully send out going posts since upgrading to 2.0.3? I get the following error when there are any outgoinmg post:


Query failed:
INSERT IGNORE INTO usenet_outgoing(poster,email,signature,newsgroup,subject,body,threadid,postid) VALUES ('tamarian','
member@lowcarber.org',,'alt.support.diet.low-carb','N.A.S. Study: Carbs turn into fat','Scientists Find Protein That T
urns Carbs Into Fat \r\n\r\nBy Emma Hitt, PhD \r\n\r\nNEW YORK (Reuters Health) - Researchers have identified a small
protein in liver cells that may help convert excess dietary carbohydrates into fat stores. They hope that the finding
will lead to the development of obesity-fighting drugs that inhibit the actions of this protein. \r\n\r\nA team led by
Dr. Kosaku Uyeda, of the University of Texas Southwestern Medical Center at Dallas, identified the protein they call
ChREBP in the liver of rats. \r\n\r\n``When people eat desserts, pasta, potatoes or other sugar- and starch-laden food
s beyond the body\'s energy and nutritional needs, these carbohydrates become a flood of glucose (sugar), and the live
r converts the surplus glucose to fat,\'\' Uyeda explained in a written statement. \r\n\r\nUyeda\'s team determined wh
ether ChREBP responded to excess dietary carbohydrates by feeding rats either a high-carbohydrate diet or a high-fat d
iet without starches. They found that the actions of ChREBP were enhanced with the high-carbohydrate diet, but not the
high-fat diet. \r\n\r\nThese actions included increasing the activity of at least two and maybe three enzymes respons
ible for making fats out of excess carbohydrates, Uyeda told Reuters Health. \r\n\r\n``Inhibition of ChREBP activation
would be expected to (lessen) excess fat accumulation resulting from a high-carbohydrate diet and provide novel oppor
tunities to address the health consequences stemming from obesity and diabetes,\'\' the researchers write in the July
31st issue of the Proceedings of the National Academy of Sciences (news - web sites). \r\n\r\n``By developing drugs to
inhibit the actions of this protein, we should slow down the conversion of excess carbohydrates to fat,\'\' Uyeda sai
d. ``There is no medication that acts that way right now.\'\' \r\n\r\nBut such a drug remains a while off, according t
o the researcher. \r\n\r\n``Now we are just beginning to understand the structure and how this protein works,\'\' he s
aid. \r\n\r\nThe next step, Uyeda added, is to isolate large quantities of the protein so that research into drug deve
lopment can begin. \r\n\r\n``But,\'\' he said, ``this may take several years.\'\' \r\n\r\nSOURCE: Proceedings of the N
ational Academy of Sciences USA 2001;98:9116-',14835,123210)

DBD::mysql::db do failed: You have an error in your SQL syntax near ''alt.support.diet.low-carb','N.A.S. Study: Carbs
turn into fat','Scientists Find' at line 1 at ./newnews.pl line 594, <SOCK1> chunk 3.
bash$

fastforward
04 Aug 2001, 01:53
Originally posted by tamarian

Paul, did you (or any usenet hack user) successfully send out going posts since upgrading to 2.0.3? I get the following error when there are any outgoinmg post:

Did you do the last usenet upgrade? This error was caused by a bug introduced when I fixed the problem of the 'not include signature' problem.

It was subsequently fixed in the last version... I think

tamarian
04 Aug 2001, 02:05
Originally posted by fastforward

Did you do the last usenet upgrade? This error was caused by a bug introduced when I fixed the problem of the 'not include signature' problem.

It was subsequently fixed in the last version... I think

Yes, I have the latest version with this signature:

$Id: newnews.pl,v 20010721 2001-07-21 03:06:32-05 paul Exp $

fastforward
04 Aug 2001, 02:14
Originally posted by tamarian


Yes, I have the latest version with this signature:

$Id: newnews.pl,v 20010721 2001-07-21 03:06:32-05 paul Exp $
Nope... that's the last but one. The latest is
# $Id: newnews.pl,v 20010729 2001-07-29 00:10:31-05 paul Exp $

I've just checked the file pointed to in the links and it definately points to the version from July 29th.

All you need to do is replace newnews.pl.

tamarian
04 Aug 2001, 02:22
Originally posted by fastforward

Nope... that's the last but one. The latest is
# $Id: newnews.pl,v 20010729 2001-07-29 00:10:31-05 paul Exp $

I've just checked the file pointed to in the links and it definately points to the version from July 29th.

All you need to do is replace newnews.pl.

Cool, thanks!

Franky
04 Aug 2001, 08:37
Hi, just like to say fantastic hack, but just one small problem. When I post on the forum, it send's the post to the newsgroup very nicely, and retrives the new posts.

But then the next time script runs, it grabs the post I've just made, so I have a duplicate post every time someone posts through the forum.

Eg: http://www.frankw.net/network/nthellworld/vbb/showthread.php?s=&threadid=1042

Thank's in advance.

-Frank

webhost
04 Aug 2001, 13:12
I ran a test post from my forum, I have a signature in my profile, just like the one attached to the bottom of this reply it works fine in vb and also on my vb board, but this is how it looks on the newsgroup server at the other end. Any idea how to correct this.
without the top and bottom lines.

also got this in results file but I never received email?

-> finding replies...
-> 2...
email notification sent to joeymoses1@home.com
Sending email to Watcher regarding reply to threadid 12547



==============================================

Joey Moses-
[bhttp://www.realwebhost.net"]Real Web Host
Reliable Web Hosting starting at $15.95 a month. Includes 24/7 Support, http://faq.realwebhost.net]
Full Knowledgebase[/url] FAQ
Real Web Host Forums! Go there! [bhttp://forums.realwebhost.net"]Forums
[bhttp://www.secureserver.net/default.asp?prog_id=RealWebHost"]Domain Names Starting @ $12.95
================================================

fastforward
04 Aug 2001, 13:24
Originally posted by Franky
Hi, just like to say fantastic hack, but just one small problem. When I post on the forum, it send's the post to the newsgroup very nicely, and retrives the new posts.

But then the next time script runs, it grabs the post I've just made, so I have a duplicate post every time someone posts through the forum.

Eg: http://www.frankw.net/network/nthellworld/vbb/showthread.php?s=&threadid=1042

Thank's in advance.

-Frank
Several posible reasons:

Your 'useragent' setting in the control panel is not set
Your 'useragent' setting in the control panel was changed between posting and retrieving
You have 're-import local posts' set to on in your control panel
When you instaled the hack, some of the SQL statements failed, ie the create unique index or add key statements.

The most likely is the last option.

fastforward
04 Aug 2001, 13:28
Originally posted by webhost
I ran a test post from my forum, I have a signature in my profile, just like the one attached to the bottom of this reply it works fine in vb and also on my vb board, but this is how it looks on the newsgroup server at the other end. Any idea how to correct this.
without the top and bottom lines.

also got this in results file but I never received email?

-> finding replies...
-> 2...
email notification sent to joeymoses1@home.com
Sending email to Watcher regarding reply to threadid 12547



==============================================

Joey Moses-
[bhttp://www.realwebhost.net"]Real Web Host
Reliable Web Hosting starting at $15.95 a month. Includes 24/7 Support, http://faq.realwebhost.net]
Full Knowledgebase[/url] FAQ
Real Web Host Forums! Go there! [bhttp://forums.realwebhost.net"]Forums
[bhttp://www.secureserver.net/default.asp?prog_id=RealWebHost"]Domain Names Starting @ $12.95
================================================
Converting of urls has always been a problem. It's very difficult to trap for every possible format. I intend to add a seperate field that will alow you to have a seperate signature purely for usenet posts (similar to the 'Public Email' option). This will mean you can create it without bbcodes and it will be displayed in usenetposts exactly as you enter it.

webhost
04 Aug 2001, 13:33
thanks any ideas on the reason why i didn't recieve a email when it said it had been sent?

fastforward
04 Aug 2001, 13:42
Originally posted by webhost
thanks any ideas on the reason why i didn't recieve a email when it said it had been sent?
Once you see the 'email notification sent to ....' message on the console, you know for sure that the sendmail function was definately called. You should have seen an error if your SMTP server did not accept your mail for delivery for delivery. Does it happen every time?

webhost
04 Aug 2001, 14:44
Sorry for the dealy Paul was running a few test.

First every test post I make from board that gets a reply to it from the newsgroup it states in results that a email is sent but it never shows up tried 5 different groups all the same.

Have you tried yours sent version 2.0.3 came out?

Pie'oh'pah
04 Aug 2001, 14:50
Hi Paul,

regarding the parse error i mentioned before, it's still there but stuff is working anyways, so what :D

Only when reading a certain group, im getting this error:

---------------------------------
Pulling alt.games.unreal.tournament.clans... No new messages.


Query failed:
UPDATE LOW_PRIORITY forum SET replycount=replycount+1 ,lastpost=996769546,lastposter='Martin H. Sluka',lastactivethread='FLOCK Fehlermeldung' WHERE forumid IN ()

DBD::mysql::db do failed: You have an error in your SQL syntax near ')' at line 1 at newnews.pl line 597, <SOCK1> line 8.
Pulling alt.games.half-life.counterstrike... No new messages. -> 41... bash-2.04$
------------------------------------

Well, somebody has stated the same error some posts earlier, but there was no reply to it so far.

This is just to let you know i have the same problem.
I suppose this support to all of us has become a full time job... No need to rush :D

-Alex

george_proost
04 Aug 2001, 15:23
hmmm...

looking at the changes.... lottasa work I think.

I got some the basics working, will wait for the upgrade notes.
Soooo many hacks to apply..
:(

webhost
04 Aug 2001, 15:37
A lot of work, the main thing they just about gutted showthread.php and moved everything to functions.php

fastforward
04 Aug 2001, 15:45
Originally posted by webhost
A lot of work, the main thing they just about gutted showthread.php and moved everything to functions.php
Don't panic. It will all be done within the next few days.

If you've already upgraded your board, the best bet is to install the hack with no code changes (exept control panel access) and set the usenet groups to read only temporarily.

DarkReaper
05 Aug 2001, 01:40
Problem with 2.0.3:

I restored everything from a backup, and got most of it installed ok, until I got to this:

4) Allow access to nntp posting date from templates

showthread.php - line 348

OLD CODE
--------
NO CODE TO REPLACE. Add the following lines directly below the line that currently reads: ' $post[posttime]=vbdate($timeformat,$post[dateline]); '

NEW CODE
--------
// START USENET HACK
$post[nntppostdate]=vbdate($dateformat,$post[nntpdateline]);
$post[nntpposttime]=vbdate($timeformat,$post[nntpdateline]);
// END USENET HACK

I couldn't find the line that reads "$post[posttime]=vbdate($timeformat,$post[dateline]);"

Stephan Whelan
05 Aug 2001, 09:33
I've been using the hack for ages now and it's been working fine.

Saying that (always a but in there somewhere) I recently upgraded to the latest version 20010721 and everything continued to work fine.

Today (after it working for a while) I went in to my database to check the usenet tables as I wanted to make sure it was all setup and optimised (as i've been upgraded since some of the early versions) and found that I was missing a couple of indexes which I added...apart from in usenet_ref table...

msgid and key are supposed to be a Primary Keys, yet I seem to have duplicates in there so cannot add it as a primary key. I'm unclear as to what the table actually does and how having duplicate entries in the msgid and key fields will affect the performance of the news pull script.

Any help here would be greatly appreciated.

NgtCrwlr
05 Aug 2001, 09:48
functions.php

Around line 80 find:

$post[postdate]=vbdate($dateformat,$post[dateline]);
$post[posttime]=vbdate($timeformat,$post[dateline]);

ADD:

$post[nntppostdate]=vbdate($dateformat,$post[nntpdateline]);
$post[nntpposttime]=vbdate($timeformat,$post[nntpdateline]);

Make sure you add the other functions.php usenet hacks also. If I remember correctly this is the only one that had to be moved

Last night I upgraded from vB 2.0.1 to 2.0.3 and managed to get usenet hack v20010714 (for vB 2.01) installed. I ran newnews.pl without problem, however, I did not test posting.

Tonight I'll tackle the latest usenet hack upgrade, with fingers crossed. :)

Good luck DarkReaper

Pie'oh'pah
05 Aug 2001, 11:19
regarding the SQL Error of mine... just forget it!

It was all my fault, since i routed the group to the wrong forum! D'oh....

Sorry 4 makin any trouble :p

fastforward
05 Aug 2001, 14:14
Originally posted by Stephan Whelan
I've been using the hack for ages now and it's been working fine.

Saying that (always a but in there somewhere) I recently upgraded to the latest version 20010721 and everything continued to work fine.

Today (after it working for a while) I went in to my database to check the usenet tables as I wanted to make sure it was all setup and optimised (as i've been upgraded since some of the early versions) and found that I was missing a couple of indexes which I added...apart from in usenet_ref table...

msgid and key are supposed to be a Primary Keys, yet I seem to have duplicates in there so cannot add it as a primary key. I'm unclear as to what the table actually does and how having duplicate entries in the msgid and key fields will affect the performance of the news pull script.

Any help here would be greatly appreciated.
This is not good. It will definately hurt performance during news loads. It won't cause duplicates in the forums though as outer joins used prior to the load prevent this.
Run the following commands:

CREATE TABLE tmp_usenet_ref AS SELECT * FROM usenet_ref;
TRUNCATE TABLE usenet_ref;
ALTER TABLE usenet_ref add PRIMARY KEY (msgid,ref);
INSERT IGNORE INTO usenet_ref select * FROM tmp_usenet_ref;
DROP TABLE tmp_usenet_ref;


PS: The latest version is usenet_gateway_v20010729__201. It looks like I forgot to update the first post in this thread last time. Sorry.

webhost
05 Aug 2001, 15:14
got any idea what this is?

NNTPERROR: 421 No Next Article
ignoring text in character set `ISO-2022-JP'
at /home/realweb/newnews.pl line 306

fastforward
06 Aug 2001, 01:55
Originally posted by webhost
got any idea what this is?

NNTPERROR: 421 No Next Article
ignoring text in character set `ISO-2022-JP'
at /home/realweb/newnews.pl line 306
I can see we need an FAQ here ;)

The first line is NOT an error. The second line is doing exactly what it's telling you. It's ignoring that particular character set because it's not what you have set in your control panel. It will still be processed by the rest of the script, but it won't have MIME conversion done on it.

fastforward
06 Aug 2001, 02:01
Release 20010805

Certified for 2.03 / instructions updated :)
Slight change in the way usenet post count is calculated on index for performance reasons.
Added support for custom usenet signature
Added option in newnews.pl to ease running on win32

Upgrading involves replacing your existing newnews.pl, replacing the usenet_settings table (make a note of your existing options), and configuring the new custom signature option if desired. Full instructions are in UPGRADE.TXT.

Download Here (http://britishexpats.com/download/usenet_gateway.tar.gz).

The latest version of this hack will always be in this first post of the thread.

DarkReaper
06 Aug 2001, 03:10
Custom profile field title for public email address
(requires additional custom profile field)
Enter the EXACT title of the custom user profile field you are using for the public email address options described above. (see instruction for more information)


I didn't see anything in the instructions about this....

webhost
06 Aug 2001, 09:34
problem in next post at bottom, but corrected

webhost
06 Aug 2001, 09:35
FROM LAST MESSAGE


Note: If the vB code edits have not been made, usenet posts WILL ALWAYS show in the results.\', \'yesno\', \'10\', \'0\', \'1\');
INSERT INTO usenet_setting VALUES ( \'35\', \'Send posts to usenet?\', \'postingenabled\', \'1\', \'This is a global switch to toggle the propogation of local posts to usenet. Posts will still appear in the local forum but will never be sent to usenet and will NOT be queued for later sending. Posts made while this switch is off will remain as local posts only.\', \'yesno\', \'5\', \'0\', \'0\');
INSERT INTO usenet_setting VALUES ( \'36\', \'Quote Size\', \'quote_size\', \'1\', \'This setting controls the text size of usenet quotes. Pixels will NOT work. The only sensible values are 1 (small) and 2 (normal).\', \'\', \'3\', \'0\', \'0\');
INSERT INTO usenet_setting VALUES ( \'37\', \'Custom profile field title for usenet signature(requires additional custom profile field)\', \'signature_profile_id\', \'Usenet Signature\', \'Enter the EXACT title of the custom user profile field you are using for the usenet signature option. (see instruction for more information)\', \'\', \'5\', \'7\', \'1\');


INSERT INTO usenet_settinggroup VALUES ( \'1\', \'Default Server Information\', \'5\');
INSERT INTO usenet_settinggroup VALUES ( \'2\', \'Article Header & Footer Information\', \'10\');
INSERT INTO usenet_settinggroup VALUES ( \'3\', \'Article Display\', \'15\');
INSERT INTO usenet_settinggroup VALUES ( \'50\', \'Script Settings\', \'50\');
INSERT INTO usenet_settinggroup VALUES ( \'4\', \'Incoming Usenet Threads & Posts\', \'20\');
INSERT INTO usenet_settinggroup VALUES ( \'6\', \'Automatic Expire\', \'35\');
INSERT INTO usenet_settinggroup VALUES ( \'5\', \'Outgoing Posts to Usenet\', \'25\');
INSERT INTO usenet_settinggroup VALUES ( \'10\', \'vB Integration\', \'40\');
INSERT INTO usenet_settinggroup VALUES ( \'9\', \'Debug & Special\', \'99\');

MySQL said: Table 'realweb_forums.usenet_setting_group' doesn't exist

This does exist

george_proost
06 Aug 2001, 09:37
TRUNCATE TABLE usenet_setting_group;

becomes

TRUNCATE TABLE usenet_settinggroup;

should solve your problem

webhost
06 Aug 2001, 09:50
Thank you George that did the trick, so today is going to be a good day after all.:)

Any idea on the email option, didn't see any instructions.

Thanks Again

fastforward
07 Aug 2001, 01:59
Originally posted by webhost
Thank you George that did the trick, so today is going to be a good day after all.:)

Any idea on the email option, didn't see any instructions.

Thanks Again
Just add a 'custom profile field' called whatever you put in your usenet control panel (the deault is 'Usenet Email Address'). Then set the 'Use custom email address' option to YES. If a user enters something in this field, that is the email that will be sent out on all usenet posts.

Similar instructions apply for the custom Usenet Signature option.

Liman Wong
07 Aug 2001, 11:48
Hi,

I really love to install this function for my board, but I have no experience with Perl and my board is using with a hosting services (it does support Perl and I know the exact Perl directory)

Is it possible to install this nice function for my board?

Thank you :D

orca
07 Aug 2001, 12:27
If you have the needed PERL modules, yes, you can use it. If you don't have the modules and your host doesn't want to install them, you can do it yourself. Just get the modules and put only the *.pm file in the correct folder. You may do a directory structure like:

/news (inside that is newnews.pl)
/news/lib

Inside /news/lib you make subdirectories named after the needed Perl Module like:

/news/lib/Date
/news/lib/Email

etc.

These directories don't have to be in the webdirectory. Now, the script needs to know where those modules are. So uncomment the line #use lib... With the directories above it should look sth like this:

use lib "/absolute/path/to/news/lib";

To know which modules are missing you can run the configured newnews.pl with perl newnews.pl at the command line. Read the error message and it wil tell what it misses.

Don't forget to chmod 755 all *.pm and *.pl files!!

Liman Wong
07 Aug 2001, 12:40
Thanks for your prompt advice. :D

DarkReaper
07 Aug 2001, 13:48
NNTPERROR: 502 You are not in my access file. Goodbye.
Connecting to news.mindspring.com... Connected
News::NNTPClient unexpected EOF on News::NNTPClient::SOCK1 at ./newnews.pl line 285
Authentication failure at ./newnews.pl line 285, <SOCK1> chunk 1.
News::NNTPClient unexpected EOF on News::NNTPClient::SOCK1 at ./newnews.pl line 0
Sending authentication info... [root@linweb01 usenet]#

???

fastforward
07 Aug 2001, 22:52
It simply means you do not have permission to access the nntp server (news.mindspring.com).

DarkReaper
08 Aug 2001, 01:36
Yea, I figured that out. Thanks though.

I'm having trouble finding news servers with lots of posts, that allow posting, and are free. Could anyone give me any suggestions for good usenet servers?

NgtCrwlr
08 Aug 2001, 02:11
I've had very good service from http://www.newscene.com/ They offer month to month at US$13/mo.

I spent a considerable amount of time experimenting with free news feeds. For me the time involved in keeping an eye on them and then searching for new ones when the one you are using decides to go off line for a week or two at time, was a pain and cost me way more than $13/mo.

On top of that Newscene knows that I'm pulling news for vBulliten and they said no problem and that they would work with me on spam issues that may arise. So far I have not allowed posting from our BBS to Usenet, I was waiting on the moderated usenet post fix from Paul. Health issues kept me from updating till now. I'm planning to upgrade to the latest unsent hack this weekend.

I'll keep everyone informed if I run into any posting problems with Newscene.com

Cheers!

fastforward
08 Aug 2001, 02:34
Originally posted by NgtCrwlr
I've had very good service from [url]So far I have not allowed posting from our BBS to Usenet, I was waiting on the moderated usenet post fix from Paul. Health issues kept me from updating till now. I'm planning to upgrade to the latest unsent hack this weekend.

Larry, in case you weren't aware, moderation support has been in for a few versions. It simply relies on the normal vB moderation process. If an outgoing usenet post is awaiting moderation, it won't be propogated to usenet.

NgtCrwlr
08 Aug 2001, 03:38
I was pretty much out of action for over a month, but I'm back in the game. I just finished upgrading to vB 2.0.3 last weekend and this coming weekend I hope to make the upgrade to the latest usenet hack.

I did get the e-mail that you sent weeks ago with instructions for adding the moderated post fix. I was out of commission and just couldn't get to it then.

I'm looking forward to this weekend. :D

DarkReaper
09 Aug 2001, 15:35
I'm having problems with messages showing up in raw form. I've set mime decoding for ISO-8859-1 and it still doesn't work:

This is a multi-part message in MIME format.

------=_NextPart_000_000A_01C120E7.4B0F1B60 Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

When I install UT Patch 436 I can't play Direct3D, because when I do, I = can't see the menus in UT
Desktop, but I can only play with OpenGL, but = then again it lagging so much I can't play, my
videocard is TNT2 Pro = with 32mb ram, I've updated the drivers but it still won't do, so I'm =
thinking of buying a new card.. Can anyone help me with this problem? I will also buy a new card,
what's the best choice between = GForce2-MX/64ram or GForce2-GS/32ram?

Thanks In Advance

Roger=20

Rogers Place=20

------=_NextPart_000_000A_01C120E7.4B0F1B60 Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> <HTML><HEAD> <META
content=3D"text/html; charset=3Diso-8859-1" = http-equiv=3DContent-Type> <META content=3D"MSHTML
5.00.2614.3500" name=3DGENERATOR> <STYLE></STYLE> </HEAD> <BODY bgColor=3D#ffffff>
<DIV><FONT face=3DVerdana size=3D2>When I install UT Patch 436 I can't = play=20 Direct3D, because
when I do, I can't see the menus in UT = Desktop, but=20 I can only play with
OpenGL, but then again it lagging so much I=20 can't play, my videocard is TNT2 Pro with
32mb ram, I've updated = the=20 drivers but it still won't do, so I'm thinking of buying a
new=20 card..</FONT></DIV>
<DV><FONT face=3DVerdana size=3D2>Can anyone help me with this=20 problem?</FONT></DIV>
<DVI><FONT face=3DVerdana size=3D2>I will also buy a new card, what's = the best=20 choice between
</FONT><FONT face=3DVerdana size=3D2>GForce2-MX/64ram or=20 GForce2-GS/32ram?</FONT></DIV>
<DVII> </DIV>
<DVIII><FONT face=3Dverdana size=3D2></FONT> </DIV>
<DIX><FONT face=3Dverdana size=3D2>Thanks In Advance</FONT></DIV>
<DX><FONT face=3Dverdana size=3D2></FONT> </DIV>
<DXI><FONT face=3Dverdana size=3D2><BR>Roger <BR><BR><A=20
href=3D"http://hjem.sol.no/andersen-media/?noframe">Rogers Place</A>=20
</FONT></DIV></BODY></HTML>

------=_NextPart_000_000A_01C120E7.4B0F1B60--

fastforward
09 Aug 2001, 20:00
Originally posted by DarkReaper
I'm having problems with messages showing up in raw form. I've set mime decoding for ISO-8859-1 and it still doesn't work:

The MIME decoder doesn't decode HTML or binary attachments. All it does is convert MIME Quoted Printable Strings back to the original form. It allows umlauts, accents and those funny Scandinavian characters to be displayed correctly.

What you're seeing is the result of users not complying with Usenet etiquette. They're probably using Netscape or Outlook to post news without bothering to configure it.

shri
10 Aug 2001, 03:08
I've got a spam filter on the term

Content-Type: text/html

Shri

DarkReaper
10 Aug 2001, 12:15
Originally posted by shri
I've got a spam filter on the term

Content-Type: text/html

Shri

Wouldn't that block the entire message? Even if half of it is in text/plain and the other is text/html? If not, how do I set it to only block the html part of the message?

Will there be HTML decoding support in upcoming versions?

DarkReaper
10 Aug 2001, 15:33
For some reason the usenet posts show up when I click get new threads and get today's active threads. I put the code below in search.php and they're still there:


$wheresql.=" AND thread.open<>10";
// START USENET HACK
if (!$includeusenetpostsingetnew) { $wheresql.=" AND thread.isusenetpost=0"; };
// END USENET HACK

AND

$wheresql.=" AND thread.open<>10";
// START USENET HACK
if (!$includeusenetpostsingetnew) { $wheresql.=" AND thread.isusenetpost=0"; };
// END USENET HACK

JCDR
11 Aug 2001, 08:59
Does anyone know any free news servers so I can do a few tests to see is everything is working a okay??

Superb hack by the way just what the doctor ordered.

JCDR
11 Aug 2001, 09:32
Hi,

The only thing left is to collect the news using the newnews.pl script. How??

Any help is very much appreciated.

webhost
11 Aug 2001, 09:55
DarkReaper

Did you deselect it in the usenet control panel options?

webhost
11 Aug 2001, 10:10
JCDR


The only thing left is to collect the news using the newnews.pl script. How??

Telnet and the best way is to setup a cron job

JCDR
11 Aug 2001, 10:13
Thanks for the reply Webhost, any chance you could post the code for it? :)

webhost
11 Aug 2001, 10:41
When using telent go to your telnet and then cd to where you have newnews.pl located at and type in ./newnews.pl then enter the script will run. for telnet you can download putty.exe here at this link http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html
or here http://www.realwebhost.net/putty.exe
when using a cron job this is my example i run a cron job every 15 minutes like this
0 * * * * /home/realweb/newnews.pl
15 * * * * /home/realweb/newnews.pl
30 * * * * /home/realweb/newnews.pl
45 * * * * /home/realweb/newnews.pl

JCDR
11 Aug 2001, 10:48
Cheers webhost,

I have putty downloaded already, I'll have a go at running the *.pl file in a moment, how do you set the cron to run automatically eveny 15 minutes or so isn't there a file I need to upload or something?

webhost
11 Aug 2001, 10:52
I have cron as apart of my hosting service, let me look around

JCDR
11 Aug 2001, 10:55
Hi WB

Sorry for all the questions :)

Below is the error returned when running the newnews.pl file via ssh

Can't locate News/NNTPClient.pm in @INC (@INC contains: /usr/libdata/perl/5.00503/mach /usr/libdata/perl/5.00503 /usr/local/lib/perl5/site_perl/5.005/i386-freebsd /usr/local/lib/perl5/site_perl/5.005 .) at ./newnews.pl line 42.
BEGIN failed--compilation aborted at ./newnews.pl line 42.

webhost
11 Aug 2001, 10:58
go to a search engine and search for cron,

The other errors that is something for fastforward.

Franky
11 Aug 2001, 13:07
Originally posted by JCDR
Hi WB

Sorry for all the questions :)

Below is the error returned when running the newnews.pl file via ssh

Can't locate News/NNTPClient.pm in @INC (@INC contains: /usr/libdata/perl/5.00503/mach /usr/libdata/perl/5.00503 /usr/local/lib/perl5/site_perl/5.005/i386-freebsd /usr/local/lib/perl5/site_perl/5.005 .) at ./newnews.pl line 42.
BEGIN failed--compilation aborted at ./newnews.pl line 42.

Installation
------------

1. Install the required perl modules:

http://search.cpan.org/search?dist=NNTPClient

PS. A 1 line crontab is: 0,10,20,30,40,50 * * * * /full/path/to/newnews.pl >/dev/null 2>&1

webhost
11 Aug 2001, 13:43
PS. A 1 line crontab is: 0,10,20,30,40,50 * * * * /full/path/to/newnews.pl >/dev/null 2>&1

The one I use will work also but I do have 1 question, what is this for Franky
>/dev/null 2>&1

fastforward
11 Aug 2001, 14:47
Originally posted by webhost


The one I use will work also but I do have 1 question, what is this for Franky
>/dev/null 2>&1
That line ensures that the actual newnews.pl output is redirected aswell as the output generated by cron. Remember, cron is just a progam that calls newnews.pl.

webhost
11 Aug 2001, 15:02
ok understand but without it I get a email for every cron job so do I even need it?

fastforward
11 Aug 2001, 15:14
Originally posted by webhost
ok understand but without it I get a email for every cron job so do I even need it?
If you don't mind getting an email for every cron job... then, no, you don't need it.

Franky
11 Aug 2001, 18:09
Yeah but who want's an e-mail for every cron job? :p Once I know it's working ok, it's pointless getting, in my case, 6 e-mails an hour telling me the same thing.

webhost
11 Aug 2001, 18:17
How would you know if for some reason it quick working and you go to your forum and find out that it hasn't downloaded for 3 hours. Meanwhile your users are expecting a answer from a post they made. Just wandering, I don't like the emails either, just wandering if you have a way that would tell us when or if it has stopped working. I was hoping you would share with us and I could turn all my emails off.

Thanks Franky

Franky
11 Aug 2001, 18:53
Well I can't personally see a way via crontab that you could have error reporting, all crontab does is to run the job and e-mail output. There is a possibility that fastforward could build error reporting into the perl script in a future version, therefore eliminating the need for lots of cron e-mails *hint hint* :)

Franky
13 Aug 2001, 19:01
I have found a problem with the script.

I post something which is shown below, snipping the irrelvent parts of the original post and quoting the relevent parts I wish to reply to, like below:

--- Begin Example ---
quoted text block 1
reply 1
quoted text block 2
reply 2
--- End Example ---

In this instance, the post on the newsgroup will read:

---
reply 2
>>quoted text block 1
>reply 1
>>quoted text block 2
---

Am I missing something here? It's completely in the wrong order, and my 1st reply is prefixed by >

PS. I got the right thread 2nd time round :p

fastforward
13 Aug 2001, 20:32
Originally posted by Franky
I have found a problem with the script.

I post something which is shown below, snipping the irrelvent parts of the original post and quoting the relevent parts I wish to reply to, like below:

--- Begin Example ---

reply 1

reply 2
--- End Example ---

In this instance, the post on the newsgroup will read:

---
reply 2
>>quoted text block 1
>reply 1
>>quoted text block 2
---

Am I missing something here? It's completely in the wrong order, and my 1st reply is prefixed by >

PS. I got the right thread 2nd time round :p
I'll need to do some testing on this.

The situation you describe only occurs when two local posts are made to a usenet post in succession and the quotes are split with the reply inserted between. It's not actually something that I've seen happen on my forums.

fastforward
13 Aug 2001, 20:33
Originally posted by webhost
When using telent go to your telnet and then cd to where you have newnews.pl located at and type in ./newnews.pl then enter the script will run. for telnet you can download putty.exe here at this link http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html
or here http://www.realwebhost.net/putty.exe
when using a cron job this is my example i run a cron job every 15 minutes like this
0 * * * * /home/realweb/newnews.pl
15 * * * * /home/realweb/newnews.pl
30 * * * * /home/realweb/newnews.pl
45 * * * * /home/realweb/newnews.pl
Don't know if you're aware, but that can be written as one sngle cron line :) ie.

0,15,30,45 * * * * /home/realweb/newnews.pl

Franky
13 Aug 2001, 20:35
Originally posted by fastforward

I'll need to do some testing on this.

The situation you describe only occurs when two local posts are made to a usenet post in succession and the quotes are split with the reply inserted between. It's not actually something that I've seen happen on my forums.

Thank you, I'd appreciate it. It would a perfect script if this could somehow be ironed out. :D

DarkReaper
14 Aug 2001, 00:41
I've got a problem with quotes also.


>blah blah blah
still in quote, but not in the quote color and lacking a > before it
>blah blah
happens again
>blah blah blah

Franky
14 Aug 2001, 01:16
Yeah but that's just a question of carriage returns and line wrapping being incorrectly parsed. I think my problem is more serious :p

DarkReaper
14 Aug 2001, 01:31
Its still a problem, and its still annoying.

fastforward
14 Aug 2001, 01:44
Originally posted by DarkReaper
I've got a problem with quotes also.


This is the fault of whatever news client was used to post the message. If you look at the message in outlook or netscape and you'll notice it's exactly the same.

Remember... this script is not intended to be a replacement of either your local forums or a real news client. It was originally written to be a simple archive of usenet posts on a subject related to your local forums.

However, outgoing posts not appearing correctly on usenet is more serious as this may result in negative feedback/complaints to your site.

Franky
14 Aug 2001, 10:33
Originally posted by fastforward

posts not appearing correctly on usenet is more serious as this may result in negative feedback/complaints to your site.
LOL, tell me about it. I got flamed for "top posting" - posting my whole reply and quoting the whole of the original message. In my example below, reply 1 is completely obscured when it's prefixed with a >

webhost
15 Aug 2001, 03:27
Request Please:

Inside of each forum setup there is a field that you set when someone starts a post in that forum that will email you or the email address you place in that field.
Request is that on the usenet forums have a field that does the same thing but only when someone post's or replies to a usenet thread on your board.

WeatherImages
15 Aug 2001, 21:07
I would absolutely love to add this hack to my message board. However, my website host provider (Site5) doesn't offer news services to clients. All of the required perl modules are installed on their servers, so the only thing lacking is a news server that I can have the script log into. Does anyone know of any good news servers out there? Are you folks using the one provided by your host or have you sought elsewhere? Thank you for the feedback:)
-Mark

fthrjack
15 Aug 2001, 23:29
i dont think the webspace provider where im working does either, the board im editing is at F2s, i cant find any details on news servers although i could be wrong they may have one, or are there any i can login to usingthis script? i dont want to add teh hack till i know it can login! cheers for any help

dwh
16 Aug 2001, 09:19
As much as I love this hack I'm a bit worried to go live with it, I've really only been testing. What happened is that I had an old version in my test server. Spent 2-3 weeks away from this hack, upgraded to vb 2.03 and came here to install the upgrade. My version was already not mentioned in the upgrade path so I had to kind of figure it out myself. I did a new install but had to manually delete a bunch of fields and now it's not quite working...on telnet I can run newnews.pl once it says "killed" and the second time it seems to work, though I haven't seen new messages posted.

There's no log file generated from cron so I think cron just gets the killed message each time. Maybe there was an issue w/ the way I upgraded but I worry that once it's production and my users get used to it, will I have to follow this thread every day to make sure I will be able to upgrade in the future? Will new releases of vb be as big a pain as this one was.

NOW PLEASE, don't get me wrong!!! This is not meant as criticism of the hack. This is the best hack EVER created for vB in my opinion. It's too good! and the support of it is too good. It gets updated so often and that's my worry, will I be out of the loop if I don't follow this thread forever?

JCDR
16 Aug 2001, 15:21
Hi Guys,

I got all the perl mod's installed but after running the newnews.pl file the following error appeared.

Can't locate MIME/Base64.pm in @INC (@INC contains: /usr/libdata/perl/5.00503/ma
ch /usr/libdata/perl/5.00503 /usr/local/lib/perl5/site_perl/5.005/i386-freebsd /
usr/local/lib/perl5/site_perl/5.005 .) at /usr/local/lib/perl5/site_perl/5.005/M
IME/Words.pm line 85.
BEGIN failed--compilation aborted at /usr/local/lib/perl5/site_perl/5.005/MIME/W
ords.pm line 85.
BEGIN failed--compilation aborted at /usr/local/lib/perl5/site_perl/5.005/MIME/W
ordDecoder.pm line 77.
BEGIN failed--compilation aborted at ./newnews.pl line 44.


Any ideas?

fastforward
16 Aug 2001, 17:28
Originally posted by JCDR
Hi Guys,

I got all the perl mod's installed but after running the newnews.pl file the following error appeared.

Can't locate MIME/Base64.pm in @INC (@INC contains: /usr/libdata/perl/5.00503/ma
ch /usr/libdata/perl/5.00503 /usr/local/lib/perl5/site_perl/5.005/i386-freebsd /
usr/local/lib/perl5/site_perl/5.005 .) at /usr/local/lib/perl5/site_perl/5.005/M
IME/Words.pm line 85.
BEGIN failed--compilation aborted at /usr/local/lib/perl5/site_perl/5.005/MIME/W
ords.pm line 85.
BEGIN failed--compilation aborted at /usr/local/lib/perl5/site_perl/5.005/MIME/W
ordDecoder.pm line 77.
BEGIN failed--compilation aborted at ./newnews.pl line 44.


Any ideas?
This indicates the MIME::WordDecoder can not be found. It comes bundled in the MIME-tools package. It may be that you have an older version of MIME::Tools that did not include the Decoder function. The latest is at http://search.cpan.org/search?dist=MIME-tools

fastforward
16 Aug 2001, 17:41
Originally posted by dwh
I did a new install but had to manually delete a bunch of fields and now it's not quite working...on telnet I can run newnews.pl once it says "killed" and the second time it seems to work, though I haven't seen new messages posted.

Check the line that reads

my $ps_pid=`/bin/ps xaww | grep newnews.pl | grep -v "grep"`;
It's near the top of the script. Make sure the script name matches what you have there. Also make sure there is no newnews.pid (or whatever you have the variable set to) is lying around in the directory.

Maybe there was an issue w/ the way I upgraded but I worry that once it's production and my users get used to it, will I have to follow this thread every day to make sure I will be able to upgrade in the future? Will new releases of vb be as big a pain as this one was.

Probably. There is no way of knowing what changes Jelsoft will make in the code and/or table structure.

Having said that, it's unlikely that any major new functionality will be added to this hack for some time. And when it does, it will be infrequently. So any changes should only be nenews.pl, which is a drop in replacement. The only other changes should be new code changes which have to made with each vB version anyway. Table structure modifications will be few and far between and only when made when necessary.

If you miss a large number of hack updates, they are all archived in the download directory. I'll change the my server permissions to allow a directory listing of the files sometime over the weekend.

Actually, there's a new version coming out tomorrow or over the weekend that fixes outgoing mail notification and 'top posting' among other things. Don't worry though, it's only a newnews.pl drop-in replacement.

JCDR
16 Aug 2001, 18:34
Hi FF

The WordDecoder.pm is there, isn't Base64.pm the file that is missing? Can't locate MIME/Base64.pm it is there but in the Decoder dir.

dwh
16 Aug 2001, 18:35
ok cool, you are the best...I'll check out what you said..

fastforward
16 Aug 2001, 18:53
Originally posted by JCDR
Hi FF

The WordDecoder.pm is there, isn't Base64.pm the file that is missing? Can't locate MIME/Base64.pm it is there but in the Decoder dir.
Just try re-installing the latest MIME-tools package and se what happens. Alternatively, you cound just copy the Base64.pm into the MIME directory.

Franky
16 Aug 2001, 21:17
Originally posted by fastforward

Just try re-installing the latest MIME-tools package and se what happens. Alternatively, you cound just copy the Base64.pm into the MIME directory.
Try following all the instructions, which includes the MIME-tools package, and then it will work. Simple.

bitbender
17 Aug 2001, 07:25
Folks, Does this hack work on VB platforms using W2K as an OS base ??

I installed this hack in my VB, after installing the latest Perl, and all went well, but I cannot seem to get newnews.pl to execute in my telnet session...It indicates that:


*===============================================================
Welcome to Microsoft Telnet Server.
*===============================================================
C:\>d:

D:\>cd webex

D:\webex>newnews.pl
Can't locate News/NNTPClient.pm in @INC (@INC contains: G:/Perl/lib G:/Perl/site
/lib .) at D:\webex\newnews.pl line 42.
BEGIN failed--compilation aborted at D:\webex\newnews.pl line 42.

D:\webex>


ok, so I did SOMEthing wrong....I went and got the lastest MSI version for Perl, and installed it b4 starting the hack...Any suggestions???


Btw, this hack looks like the 'gold', once I get it working on my board, I've visited some of the samples listed in this thread!!

Any help would be appreciated!! (and if this seems a silly request, it's late here!)

Franky
17 Aug 2001, 11:26
I find it amazing how many people do not READ the instructions.

Installation
------------

1. Install the required perl modules:

http://search.cpan.org/search?dist=NNTPClient
http://search.cpan.org/search?dist=libnet
http://search.cpan.org/search?dist=MIME-tools
http://search.cpan.org/search?dist=Text-Autoformat
http://search.cpan.org/search?dist=MailTools
http://search.cpan.org/search?dist=URI-Find
http://search.cpan.org/search?dist=DBI
http://search.cpan.org/search?dist=Msql-Mysql-modules
http://search.cpan.org/search?dist=TimeDate
http://search.cpan.org/search?dist=HTML-Parser

orca
17 Aug 2001, 12:02
Well, Franky, if you ever worked in a Computer Store you wouldn't be amazed. That's your daily business. People usually run to the store b4 reading...

JCDR
17 Aug 2001, 12:10
Hi Franky,

All instructions were read, and the modules were installed by our hosts using the links you have posted.

Should Base64.pm be in the Decoder dir (which it is in) or the MIME dir?

bitbender
17 Aug 2001, 14:08
Thanks for your reply, but could I have it w/o the attitude?

Haven't you ever been 'new' to something? or make a simple mistake?

I read it, I printed each doc section, I backed everything up, and checked the changes i DID make.

Please dont assume EVERYONE understands EVERYTHING that you post here. If you were new in my shop, I would not have spoken to you in that fashion, and I mentor lots of newbies daily.

Some of us come from very diverse backgrounds. You know, the Perl MSI doc indicated it 'installed all necessary components', I was naive enough to believe it. Now I won't.

Thanks for the reply. As I stated before, this is a great piece of work.

Franky
17 Aug 2001, 14:46
Originally posted by JCDR
Should Base64.pm be in the Decoder dir (which it is in) or the MIME dir?
In the Decoder dir mate.

bitbender: sorry, bad day :o Is it working okay for you now?

JCDR
17 Aug 2001, 14:50
Cheers Franks, thats where it is looks like I'll have to do a fresh install myself this time to see if it resolves the problem.

Franky
17 Aug 2001, 15:32
You could always try installing the files in a local directory, that's what I did, I couldn't be bothered to hassle the hosts to install them, so I just put them in a dir inside my home dir.

bitbender
17 Aug 2001, 16:49
I understand bad days! No problems! I was probably a "bit" harsh, I'd spent too much time with computers on thursday!

I will post my results later after I make sure I have all my "pieces-n-parts" aligned in neat little piles!! As a career programmer, I hate bothering people when I haven't done my piece correctly.

But for now, I must pay homage to the mainframe deities that make my forum hosting services possible(my mainframe job). Hence, I am prevented from doing interesting work for the next 6 or 7 hours....

The VB hacks(related to this function) are all quite functional, and I have done integration testing with the legacy functionality, and all is working fine. Just my omissions causing the newnews.pl script to fail appear to be my only issues in getting this functional.

I am sort of confused, because I cannot tell where W2K installed the Perl source base. It was 37meg, so you'd think a guy could find a trail somewhere..but it was late, and the toothpicks hiolding my eyelids up were becoming quite painful.....I'll probably find it straightaway later today!!

cheers!

fastforward
17 Aug 2001, 18:42
bitbender,

James Cridland posted this a while ago giving instructions for win32 users. It might be helpful.

1. [Installing the modules]
WIN32/ActivePerl: To install these, type "PPM" into a command box. Then, type:
install News-NNTPClient
install NetServer-SMTP
install DBD-Mysql
install Text-Autoformat
install TimeDate
install MIME-tools
install URI-Find

Remember to set the iswin32 variable to 1 in newnews.pl before running it.

bitbender
17 Aug 2001, 20:16
Bless you FastForward!! And say hey to my Dad, he lives in Forrest City, N. C!! Don't know how close you are to that place, but it's nice there!

Hey, was the above information in the 1st thread about this hack? I have been following this thread since I saw the big red banner in the first one that directed me to this thread! No matter, really, I just wanted to make sure I was reading all the pertinent and neccessary data!

THANK YOU for responding...I've been reading all about perl for several hours now, and had just about decided to go and become a Tibetian monk, and sit on a mountain in the himalaya's!

Whew!~ I was afraid I'd be making packages until the cows came home....Can't wait to get back to the forum center to try it !!

Thanki, thanki, thanki, thanki....(did you ever watch Gomer Pyle??) :D :D :D :D

Update:

FastF, totally AWSOME results from your post, I mean inspiring!!

Working fine now! Tanks again!

webhost
17 Aug 2001, 23:17
Fastforward

All fixed I had a total of 2369 usenet post's in the psot table even though I deleted all of them, they showed in th db as having userid zero, so Kier gave me this command to run and they are removed and usenet count is set to zero to restart. This is the command he gave me,

DELETE FROM post WHERE userid=0




This ended working out pretty good by doing it I cut the length of my opening page 40%.

My forum here http://forums.realwebhost.net
Usenet at bottom

James Cridland
18 Aug 2001, 21:20
Originally posted by fastforward
However, by doing that, you have established the problem is probably with the news server and not the script. Do you have access to another news server you can try? If not, PM me your IP and I will give you temporary access to my DNews server for testing.

After a re-install of perl onto a different machine, so that the main server doesn't have to cope with the processor load, I'm now running the gateway again. (Quite disappointed to see that my documentation for W32 didn't make it into the latest release).

Anyway, my speed problem - up to 90 seconds per retrieved post - seems to be down to my local ISP's news server at news.btinternet.com... I've tried testing the script with news.grc.com, a public news server dedicated to internet security, and it gets approximately two new messages a second, which looks quite acceptable in comparison!

This is Good News, up to a point: since there are no free NNTP servers for W32 that I can find, I still need to work on sourcing a new news server to use.

Wonder what the problem is?

webhost
19 Aug 2001, 11:14
Small problem

My cron job which as ran ok for awhile is having problems even though it is setup the same way. Not getting any data
So I went to telnet and ran manually.
the first time I try to run the script from telnet, I enter it and it comes back and says killed, I run it again and it works fine.
It does the same way everytime, it fails the first saying killed but always runs the second time. Any ideas what is causing this?

Franky
19 Aug 2001, 17:20
Originally posted by fastforward
Actually, there's a new version coming out tomorrow or over the weekend that fixes outgoing mail notification and 'top posting' among other things. Don't worry though, it's only a newnews.pl drop-in replacement.
/me really want's to see this :D

fastforward
20 Aug 2001, 05:26
Release 20010820
more tweaks to newnews.pl.

Rewrote indexing routine to be much faster & more efficient
Fixed outgoing email notification
Takes some advantage of InnoDB and Gemini transactions if available
Outgoing posts no longer top post
Lots of other little tweaks
To upgrade, you just need to replace your existing newnews.pl with the one in this package (http://britishexpats.com/download/usenet_gateway.tar.gz).

All links in every post point to the latest version of this hack.

Franky
20 Aug 2001, 11:50
Eeek something wrong here.

I post a quoted reply, and I get 2 posts (same timestamp)

1.
---
Frank (/i)
(B)test (/B)(/QUOTE)
test
(QUOTE)(i)Originally posted by Frank wrote:
> test


part 2
---

2.
---
Frank wrote:
> test


test again
---

The second appears correctly, but what is the 1st one doing on the newsgroup? PS. Square brackets replaced with round for purpose of this post.

dwh
20 Aug 2001, 17:28
One suggestion, I think creating a whole new style for this hack is a bit of overkill. It doesn't work for me because I keep changing templates. I think some modification of showthread.pl should be possible to make the posts show up nicer w/o having to create a brand new style. It'll probably be a couple of weeks but I'll look at the code soon.

fastforward
20 Aug 2001, 22:36
Originally posted by dwh
One suggestion, I think creating a whole new style for this hack is a bit of overkill. It doesn't work for me because I keep changing templates. I think some modification of showthread.pl should be possible to make the posts show up nicer w/o having to create a brand new style. It'll probably be a couple of weeks but I'll look at the code soon.
Yeah, I toyed with the idea of this but I was concerned with the extra support these changes might need. I've tried to minimize the code changes (even if they are optional). I've been hoping that Jelsoft will change the way templates, styles and replacements are all tied together as one entity. If they could be referenced seperately, it would make things a lot more flexible. You might want to look at the 'conditionals in templates' hack that Ed Sullivan posted.

dwh
20 Aug 2001, 22:46
heh heh, yup seems we're on the same wavelength...

Brian Cruz
22 Aug 2001, 07:13
I've just installed this hack and it's working very well so far. I have a question about the spam control feature. All the options there are a bit confusing to me, and I'm not quite sure how to make it do what I want. For example, if I didn't want to download a message with "XXX" or "$$$" in the title, how would I set that up? What about a post that had the text "Britney Spears sexy pics" in the body, not the title? What exactly does "treat as regular expression" do?

Also, in the vb_code_changes.txt, is the list of changes wrong? I don't see any instructions for change #5.

btp*
22 Aug 2001, 14:08
Questestions

1, does it add alot if size to the database

2, why can't i download it, it says

Forbidden
You don't have permission to access /download/usenet_gateway.tar.gz on this server.


--------------------------------------------------------------------------------

Apache/1.3.20 Server at britishexpats.com Port 80

webhost
22 Aug 2001, 14:19
did you change file to 755

fastforward
22 Aug 2001, 17:13
Originally posted by btp*
Questestions

1, does it add alot if size to the database

depends on how many newsgroup posts you download.

2, why can't i download it, it says

Forbidden
You don't have permission to access /download/usenet_gateway.tar.gz on this server.

That was my fault. I didn't add the 'FollowSymlinks' directive to the Apache conf.

fastforward
22 Aug 2001, 17:21
Originally posted by Brian Cruz
For example, if I didn't want to download a message with "XXX" or "$$$" in the title, how would I set that up?

enter XXX in to the Expression field.
select 'Header' in the Scope drop down.
leave everything else as the default ands click submit


What about a post that had the text "Britney Spears sexy pics" in the body, not the title? What exactly does "treat as regular expression" do?
As above, but select 'Body' from the drop down. Not sure why you'd want to block sexy pics of Britney though :D A regular expression is a 'mask' or 'wildcard'. You can buy complete books on regular expressions. This script uses Perl type regexes.

Also, in the vb_code_changes.txt, is the list of changes wrong? I don't see any instructions for change #5.
Yes, the contents list is wrong.

webhost
22 Aug 2001, 17:39
I have purged all newsgroups and started over 3 days ago, now total
posts are off again compared to the number of total posts shown in the newsgroups. by 3000.

What would be the fix for this?

JCDR
22 Aug 2001, 17:43
After getting all the modules installed, I ran the newnews.pl file via ssh and got the following??

Killed


?? any ideas?

fastforward
22 Aug 2001, 18:14
Originally posted by webhost
I have purged all newsgroups and started over 3 days ago, now total
posts are off again compared to the number of total posts shown in the newsgroups. by 3000.

What would be the fix for this?
No idea. It doesn't happen on my forums.

fastforward
22 Aug 2001, 18:15
Originally posted by JCDR
After getting all the modules installed, I ran the newnews.pl file via ssh and got the following??

Killed


?? any ideas?
Go back a few pages. There's a couple of posts on this. Make sure you haven't got an old pid file hanging around in the directory.

webhost
22 Aug 2001, 18:24
No idea. It doesn't happen on my forums.

What is the chance of you looking at the way I have my forums setup
to see if there is something wrong there?

JCDR
22 Aug 2001, 18:24
Thanks FastForward,

I did do a search for killed but trying to find the message is like trying to find a needle in a umm......... :)

All sorted now thanks, for some reason the newnews.pl file had newnews2.pl instead of newnews.pl on that line.

fastforward
22 Aug 2001, 18:36
Originally posted by JCDR
Thanks FastForward,

I did do a search for killed but trying to find the message is like trying to find a needle in a umm......... :)

All sorted now thanks, for some reason the newnews.pl file had newnews2.pl instead of newnews.pl on that line.
Ahh.. that would be my fault. newnews2.pl is the name of the script I used for testing. Sorry about that. :o I'll change it to automatically grab the script name at runtime in the next release.

fastforward
22 Aug 2001, 18:44
Originally posted by webhost
No idea. It doesn't happen on my forums.

What is the chance of you looking at the way I have my forums setup
to see if there is something wrong there?
I did look. The only difference is the number of levels.

Brian Cruz
22 Aug 2001, 19:16
Thanks for your help with the spam filter. I've noticed that the script (when running as a cron job) occasionally misses a few posts made to a group. If I purge the group and set last msg to 0, then run the script manually, it downloads what it missed before. Any idea why that might be?

Franky
22 Aug 2001, 20:35
Originally posted by Franky
Eeek something wrong here.

I post a quoted reply, and I get 2 posts (same timestamp)

1.
---
Frank (/i)
(B)test (/B)(/QUOTE)
test
(QUOTE)(i)Originally posted by Frank wrote:
> test


part 2
---

2.
---
Frank wrote:
> test


test again
---

The second appears correctly, but what is the 1st one doing on the newsgroup? PS. Square brackets replaced with round for purpose of this post.
Uhm does anyone have a solution for the above problem...? My usenet forums are disabled at the moment due to this. Thanks.

webhost
22 Aug 2001, 20:54
I did look. The only difference is the number of levels.

FastForward

What is the chance of you glancing at my forum level setup in my control panel. Been trying to resolve this since last week. Would like to put problem to bed if you will help. I believe it to be something simple.

fastforward
23 Aug 2001, 02:20
Originally posted by Franky

Uhm does anyone have a solution for the above problem...? My usenet forums are disabled at the moment due to this. Thanks.
I haven't been able to duplicate this problem yet.

JCDR
23 Aug 2001, 10:36
Hi Guys,

More problems?? :( :)

#####
NNTPERROR: 502 Permission Denied - Access not permitted (cable modems please use news.cable.ntlworld.com) - newsmaster@ntli.net (Typhoon v1.2.3)
Connecting to news.ntlworld.com:119... Connected
News::NNTPClient unexpected EOF on News::NNTPClient::SOCK1
at ./newnews.pl line 340
Authentication failure at ./newnews.pl line 340, <SOCK1> chunk 1.
News::NNTPClient unexpected EOF on News::NNTPClient::SOCK1
at ./newnews.pl line 0
######

Any ideas on what could be causing the above error

Franky
23 Aug 2001, 10:40
Is it not obvious? 502 Permission Denied - Access not permitted. It even tells you what to do, so why don't you just change the address of the server to the one it tells you?

JCDR
23 Aug 2001, 10:44
Hi Franky,

I did try news.cable.ntlworld.com but the same error appeared, I have included a ntl user and pass.

orca
23 Aug 2001, 10:56
Is your forum on your PC at home and you connect by cable? If your forum is on a hosted server it won't use cable. And if it's on a hosted server, it could be that your news host only supports IP's coming from their IP range (I'm assuming that it's your internet provider you try to get the news from).

JCDR
23 Aug 2001, 11:01
Hi orca,

The site is on a dedicated server - with a host -, and yes it is an ISP I am trying to get the news from, I presume franky uses ntl for his newsgroups what setting do you use mate?

orca
23 Aug 2001, 12:21
I use a free newsserver without any passwords. Found it by doing a search for free newsserver in google.

webhost
23 Aug 2001, 12:32
orca

Can you write to that free sever?

JCDR
23 Aug 2001, 17:08
Thanks for all you help over the last few weeks guys, I would have gone out of my mind without you :) it all seems to be working okay down on the test forums using a free newsserver all I need now is for franky to tell me the set-up for ntl? :) :) please :)

bitbender
24 Aug 2001, 00:14
In my travels with Google, I have located several free servers. I did, however heed the advice given by J. Cridland in this thread. And I have experienced stabilty issues wereas they are not up all the time.

On the other hand...they were instrumental in helping me get going with this hack, and so are good for that purpose. The ones I found allow posting, but tend to have alot of spam (or so say my filters!).

I ended up spending the small amount of change for a paid service, and the news server that my broadband provider supplies. Great results!

If any one wants the small list I have compiled, please PM me, or email me, I will be glad to give 'em up. I'd post 'em here, but I don' know if that's a violation of the rules.

Once again, FastF, this is about the coolest thing for the board I could think of! My membership (albeit small) is already raving over the effect's!! If I could buy you a pint, I'd ship you a case!!

From one programmer to another, very nice piece of work!

Pie'oh'pah
24 Aug 2001, 01:43
Hey Paul,

Yer hack still workin fine with me.

I have a suggestion though:
What about makin it possible, when searching for new posts, to display not all new usenet messages, but those messages, where board members have participated in?

Any chance?

-Alex

JCDR
24 Aug 2001, 11:24
When a user of our forums post's a reply to the usenet group, how long will this take to show up in the usenet group (ie not on the forums in outlook express for example) As I posted a test message but the message has yet to be posted to the usenet group but has shown up on our forums.

any ideas?

fastforward
24 Aug 2001, 14:51
Originally posted by JCDR
When a user of our forums post's a reply to the usenet group, how long will this take to show up in the usenet group (ie not on the forums in outlook express for example) As I posted a test message but the message has yet to be posted to the usenet group but has shown up on our forums.

any ideas?
It goes out the next time you run the script. How long it takes before it shows up on the news server is dependent on each server. Assuming the post was accepted, and you are reading from the same server to which you posted, it should be only a few minutes.

fastforward
24 Aug 2001, 14:58
Originally posted by Pie'oh'pah
What about makin it possible, when searching for new posts, to display not all new usenet messages, but those messages, where board members have participated in?
Yeah, that should be easy. Usenet posts made by forum members can be identified by a '-1' in the isusenetpost column in the post table.

It might take me a few weeks to get around to it though. I'm in the middle of re-doing BritishExpats.com. It's still running vB 1.1.6 so I think it's about time I did something about it. :p

You can make the change yourself if you like. It's easy peasy. Find the line that was added by the usenet hack in search.php. It should read:
$wheresql.=" AND thread.isusenetpost=0";
Change that to:
$wheresql.=" AND thread.isusenetpost=0 AND post.isusenetpost <= -1";

If you just want threads that were started by members, change it to:
$wheresql.=" AND thread.isusenetpost <= -1";


bitbender:
Thanks for your kind words. :D

Brian Cruz
24 Aug 2001, 20:15
Originally posted by fastforward

It goes out the next time you run the script. How long it takes before it shows up on the news server is dependent on each server. Assuming the post was accepted, and you are reading from the same server to which you posted, it should be only a few minutes.

I had waited 24 hours and the posts never showed up, but after making a post and running the script manually, the post would appear in only a few minutes (the older ones were still MIA). I figured out how to restart my crond daemon, and the last post I made went through, so maybe that was the problem.

I've also noticed that when outgoing posts which contain quotes appear on usenet, the quoted text has blank lines between them. They look like this:

Juan F. Lara wrote:
> Years ago I heard that Warner Brothers Television Animation was going to

> make an "Osmosis Jones" TV series to go along with the movie. Whatever

> happened to that TV series? Are they still going to make it?


Is that normal for the script or did I set up something wrong?