Slimelite BBS

orig MSGID -> new REPLY

Back to FREE25

From Pierre Phaneuf · 1996-04-07 03:45 · #73109 in FREE25

In reply to Re: orig MSGID -> new REPLY by HECTOR SANTOS

 HS> However, the last sentence says, "the manner in which this serial
 HS> numner is generated is left to the implementor".

 HS> You can now understand that FTS-9 invalidates the principle of a
 HS> standard being implemented across all systems in order to guarantee
 HS> uniqueness across systems.

Why? When comparing MSGID, you are supposed to take into account the whole string. If you compare only the serial number, I can understand your point of using a CRC to produce it! But as long as the number is unique on my system, you can stop caring about how I produced it, just be sure there is no identical serial number coming from 1:167/136.14.

 HS> In order to implement any standard MSGID function generator, the
 HS> FTS-9 specification must be changed. If it must be changed, then we
 HS> should alter all it flaws and not just use an 8 digit number.  We can
 HS> also address the ADDRESS factor as well.

Address the ADDRESS factor? Look at the beginning of the MSGID... What do we have here? An *address*, wow! :-)

 HS> Practical Engineering Point #2:  We can only guarantee that two
 HS> messages are not the same for your OWN system. You can not guarantee
 HS> this on others.

And it is perfect that way. I take care to check for both the address string and the serial number. Consider those:

^AMSGID: 1:167/136.14 0a3158af
^AMSGID: 1:167/309 0a3158af

Would your system consider those as dupes? Not mine... And those:

^AMSGID: 1:167/136.14 b3a45fac
^AMSGID: 1:167/136.14 b3a45fac

Would discard one of them without any further processing. No need to look at *anything* else than the MSGID, when it's there. (but you assumed this in your example)

 HS> Lets just state for the sake of simplicity that the magic number is
 HS> 500 messages per conference, after all, MOST BBS software place a
 HS> default value by default.  500 is high but this is just an example.

I use a binary tree working by echo. I use dynamically managed memory and a MSGID with a 25 byte address (rough average) takes about 37 bytes. A 500 message echo storing a complete b-tree and assuming 25 bytes addresses, 2 pointers 4 byte long and the serial number stored as a dword, uses 18500 bytes of memory. Do you have a problem with that? I don't even look at the PATH line, as it is not useful at all. Where I *do* have a problem is when I encounter a message without a MSGID. I use a CRC of the *whole* message coupled with the origin address. Quite safe. And I don't store the dupe table, I use the working message base to built my b-tree on the fly.

 HS> Practical Engineering Point #4:  Do not assume the system will
 HS> support or the sysop "honor" the theoritical load requirements for
 HS> FTS-9 dupe checking requirements.

How about 0 byte?

 HS> What I do is check for the serial # and compare it in my dupe table.
 HS> But note this voilates my "Practical Engineering point #2" which says
 HS> I am NOT aware of the other guys serial number generation formula.

And you don't need to. If you compare the whole thing as I do, you won't get false dupes when comparing two messages that have MSGID.

 HS> One may also argue, "But Hector, thats the whole point of this
 HS> thread,
 HS> lets develop that formula".  I agree, but refer to my "Practical
 HS> Engineer Point #1".  Unless there is across board compliance, you can
 HS> never rely on the "other guys" formula.  And even if there was a
 HS> decent PROPOSAL made, you can not enforce it. So you will always be
 HS> stuck with generating your own number for dupe checking.

In any case, I don't care much about how you actually generate the number. The problem is that if you *do* generate one using a method (namely CRC 32) that could produce two identical MSGID (both the address and the serial number), one of them will be dumped by my system. Use a bulletproof algorithm, and everything will work like it was on bearings.

 HS> But remember, EVEN if you do find one, it isn't a DUPE unless the
 HS> ADDRESS comparisions are made in the MSGID and PATH. In other words,
 HS> if you find a message in echo X and message in echo Z with the same
 HS> CRC, you must also systematic do the preliminaries comparisions
 HS> before
 HS> you conclude erroneous duplicity.

The whole point of MSGID is to optimize dupe elimination by having a single lookup to do. Basically, I could just ignore the MSGID and calculate my own CRC32 all the time and so be it. But why support MSGID then? When it's there, use it.

Pierre
phaneuf@absint.login.qc.ca

--- FMail/386 1.02
 * Origin: Zaphod Beeblebrox for President! (1:167/136.14)