From Pierre Phaneuf · 1996-04-07 03:45 · #73109 in FREE25
In reply to Re: orig MSGID -> new REPLY by HECTOR SANTOS
HS> However, the last sentence says, "the manner in which this serial HS> numner is generated is left to the implementor". HS> You can now understand that FTS-9 invalidates the principle of a HS> standard being implemented across all systems in order to guarantee HS> uniqueness across systems. Why? When comparing MSGID, you are supposed to take into account the whole string. If you compare only the serial number, I can understand your point of using a CRC to produce it! But as long as the number is unique on my system, you can stop caring about how I produced it, just be sure there is no identical serial number coming from 1:167/136.14. HS> In order to implement any standard MSGID function generator, the HS> FTS-9 specification must be changed. If it must be changed, then we HS> should alter all it flaws and not just use an 8 digit number. We can HS> also address the ADDRESS factor as well. Address the ADDRESS factor? Look at the beginning of the MSGID... What do we have here? An *address*, wow! :-) HS> Practical Engineering Point #2: We can only guarantee that two HS> messages are not the same for your OWN system. You can not guarantee HS> this on others. And it is perfect that way. I take care to check for both the address string and the serial number. Consider those: ^AMSGID: 1:167/136.14 0a3158af ^AMSGID: 1:167/309 0a3158af Would your system consider those as dupes? Not mine... And those: ^AMSGID: 1:167/136.14 b3a45fac ^AMSGID: 1:167/136.14 b3a45fac Would discard one of them without any further processing. No need to look at *anything* else than the MSGID, when it's there. (but you assumed this in your example) HS> Lets just state for the sake of simplicity that the magic number is HS> 500 messages per conference, after all, MOST BBS software place a HS> default value by default. 500 is high but this is just an example. I use a binary tree working by echo. I use dynamically managed memory and a MSGID with a 25 byte address (rough average) takes about 37 bytes. A 500 message echo storing a complete b-tree and assuming 25 bytes addresses, 2 pointers 4 byte long and the serial number stored as a dword, uses 18500 bytes of memory. Do you have a problem with that? I don't even look at the PATH line, as it is not useful at all. Where I *do* have a problem is when I encounter a message without a MSGID. I use a CRC of the *whole* message coupled with the origin address. Quite safe. And I don't store the dupe table, I use the working message base to built my b-tree on the fly. HS> Practical Engineering Point #4: Do not assume the system will HS> support or the sysop "honor" the theoritical load requirements for HS> FTS-9 dupe checking requirements. How about 0 byte? HS> What I do is check for the serial # and compare it in my dupe table. HS> But note this voilates my "Practical Engineering point #2" which says HS> I am NOT aware of the other guys serial number generation formula. And you don't need to. If you compare the whole thing as I do, you won't get false dupes when comparing two messages that have MSGID. HS> One may also argue, "But Hector, thats the whole point of this HS> thread, HS> lets develop that formula". I agree, but refer to my "Practical HS> Engineer Point #1". Unless there is across board compliance, you can HS> never rely on the "other guys" formula. And even if there was a HS> decent PROPOSAL made, you can not enforce it. So you will always be HS> stuck with generating your own number for dupe checking. In any case, I don't care much about how you actually generate the number. The problem is that if you *do* generate one using a method (namely CRC 32) that could produce two identical MSGID (both the address and the serial number), one of them will be dumped by my system. Use a bulletproof algorithm, and everything will work like it was on bearings. HS> But remember, EVEN if you do find one, it isn't a DUPE unless the HS> ADDRESS comparisions are made in the MSGID and PATH. In other words, HS> if you find a message in echo X and message in echo Z with the same HS> CRC, you must also systematic do the preliminaries comparisions HS> before HS> you conclude erroneous duplicity. The whole point of MSGID is to optimize dupe elimination by having a single lookup to do. Basically, I could just ignore the MSGID and calculate my own CRC32 all the time and so be it. But why support MSGID then? When it's there, use it. Pierre phaneuf@absint.login.qc.ca --- FMail/386 1.02 * Origin: Zaphod Beeblebrox for President! (1:167/136.14)