Loot Items with missing IDs

[Belissario]

Greetings,

been looking at the recent (as of U8) loot from the "Inn of the Forsaken" and both "Tham Mirdain" instances. The problem is that the items are generic and no ID can be obtained via Compendium, Tulkas or other tools. The Shortcut cannot provide a fixed ID. Am I doing something wrong and is there a workaround?

An example of what I'm talking about: http://lorebook.lotro.com/wiki/Armou...on's_Cloak

[Feoktist]

Greetings,

No idea why this has been posted under my other id.. just to make it clear, it's me.

[Fredelas]

Greetings,

been looking at the recent (as of U8) loot from the "Inn of the Forsaken" and both "Tham Mirdain" instances. The problem is that the items are generic and no ID can be obtained via Compendium, Tulkas or other tools. The Shortcut cannot provide a fixed ID. Am I doing something wrong and is there a workaround?

An example of what I'm talking about: http://lorebook.lotro.com/wiki/Armou...on's_Cloak

I think the kind of items you're looking at are "scaling items". (I don't know the real name for them.) The item level and resulting stats for these items aren't fixed until the items actually drop from a scaled instance.

I'm not sure how they get handled in terms of shortcuts. When players look at tooltips for these items when they're linked to chat in-game, the results seem to vary widely.

According to the Lorebook, the item you linked should have an ID of 0x7002928A. Is there anything like that in its data?

[Belissario]

According to the Lorebook, the item you linked should have an ID of 0x7002928A. Is there anything like that in its data?

There seems to be a change in behavior since all equip IDs are null. Tulkas has a box where you can drop an item to see its ID. Regardless of what item I try to drop I invariably get an error "Tulkas: Item contains an invalid ID: 0."
So I went on and looked into the value that is provided by GetShortcut():GetData() method and here's a few observations:

Jacket of the Hytbold Knife: "0x03AB0002B00DC782,0x00000000 " - Tulkas has ID 1879247103 (0x700308FF)
Jacket of the Hytbold Gambler: "0x03AB0002C175F5AF,0x00000000 " - Tulkas ID 1879247107 (0x70030903)
Bracer of the Huntsman: "0x03AB0002B0B022FC,0x00000000 " - Tulkas ID 1879216990 (0x7002935E)
Agile Bracelet of Éomer: "0x03AB0002B89CFAD4,0x00000000 " - Tulkas ID 1879243628 (0x7002FB6C)
Loop of the Dunlending Archer: "0x03AB000225E5AA0D,0x00000000 " - Tulkas ID 1879217749 (0x70029655)
Empowered Anduin Vagabond's Earring: "0x03AB0002597D8246,0x00000000 " - Tulkas ID 1879232308 (0x7002CF34)
Skillful Dagger of Éomer: "0x03AB0002B088F6DE,0x00000000 " - Tulkas ID 1879243740 (0x7002FBDC)
Creoth Sacrificial Blade: "0x03AB0002B0716928,0x0000 000" - Tulkas ID 1879217424 (0x70029510)

BUT at the same time
Rohan's Celebrant Salve: "0x03AB0002C2848511,0x70030F41 "
Draigoch's Scale: "0x03AB0002C333EB8F,0x7002510B "

That led me to think that once you equip an item it looses it's short ID but no, any equippable item now has a short ID of 0x00000000. Also, all the long IDs I've checked start with 0x03AB0002 for me. I see a comment in Tulkas code that looks like "0x03240002AD311046,0x70027710 " - so probably "0xAB" means Russian and "0x24" means English somehow.

Now the question is whether the remaining 4 bytes of the long code can be converted into the old ID. Before I go deeper into that, can you please confirm you're seeing the same?

[Fredelas]

Item instance IDs may change every time you log out and back in. I suspect this data is in some packed binary format, maybe not aligned on byte boundaries. It may be necessary to expand it completely bitwise to discover what information it contains.

Of note, your experiment seems to show that stackable items maintain a clear reference to the item's data ID (possibly to aid the display of multiple stack quantity totals in shortcuts), whereas non-stackable items don't keep this extra bit of information.

Edit: I can't find any reference from an item instance ID to the data ID of the base item it represents. Perhaps this relationship is now only stored on the server and is queried every time an item's name or icon image is needed in the client. It might be worth capturing the the item instance examination tags from chat events when they're linked to chat to see if they contain any more data.

From your first example (Jacket of the Hytbold Knife):

Your item instance ID from your shortcut: 03 AB 00 02 B0 0D C7 82

00000011 10101011 00000000 00000010 10110000 00001101 11000111 10000010

Data ID that instance represents (not in shortcut): 70 03 08 FF

01110000 00000011 00001000 11111111

[Feoktist]

Item instance IDs may change every time you log out and back in. I suspect this data is in some packed binary format, maybe not aligned on byte boundaries. It may be necessary to expand it completely bitwise to discover what information it contains.
...
Edit: I can't find any reference from an item instance ID to the data ID of the base item it represents. Perhaps this relationship is now only stored on the server and is queried every time an item's name or icon image is needed in the client. It might be worth capturing the the item instance examination tags from chat events when they're linked to chat to see if they contain any more data.
...

For the record, after 1 relog that magic number didn't change.

I traced how the items are linked to the chat by default Ctrl+RMB hoping to get more insight, but instead discovered that the Jacket of the Knife is linked via

<ExamineItemInstance:ItemInfo: here comes a sequence of about 140 bytes>[Jacket of the Hytbold Knife]<\ExamineItemInstance>

This sequence of bytes (shown in chat as 72 UTF-8 symbols, didn't look at it in binary form yet) also persisted through relog. Probably it has item ID somewhere, but finding it will not be an easy task =( Moreover, if the same type of item is linked by someone else, I can see a shorter sequence, about 120 bytes long.

Seems like the only valid source for grabbing the item IDs are the <Examine:IIDDID:...> links one can get at vendors and chests.

Interesting that Legendary Items are linked as <ExamineIA:IAInfo: here comes 400+ bytes line>[LI Name]<\ExamineIA>. The sequence is considerably longer than for regular items, the legacy info might be encrypted there somewhere.

[Garan]

The first item ID in the shortcuts are a Server unique ID for an item or stack of items. The second ID is the generic item ID (when it isn't 0). The unique IDs are generated sequentially any time the server needs to reference a new instance of an item or stack. For instance, loot a node or chest when you have no instances of the items the node/chest could drop. Then check the item IDs for those items and the unique ID will be sequential (assuming the node/chest had multiple items). There does seem to be at least two pools of IDs being allocated since loot from killing mobs drop items that are sequential from one pool while items gathered from nodes are sequential from a slightly different pool. The first part of each of those unique IDs reference the ID type and server you are on, for instance 0A is Landroval, 04 is Meneldor, 16 is Snowbourn, etc. and the 03 is for Items. Those same server IDs appear in several other places such as the select object IDs that contain the player names in chat messages (so 030A is a Landroval Item while 020A is a Landroval Character) .

Example:
A low level mob dropped these three items:
0x030A00039929C66E,0x7000DB17
0x030A00039929C66F,0x7000DCBC
0x030A00039929C670,0x70001E05
then a node immediately harvested after killing the mob dropped these items:
0x030A0003992C0DD7,0x7000394C
0x030A0003992C0DD9,0x70004EBE
then another low level mob dropped these:
030A00039929C6A6,0x7000DB17
030A00039929C6A7,0x70001E05
and a third mob dropped:
030A00039929C6AD,0x7000DCBC

It should be readily apparent that the mob drops are being assigned unique IDs from one pool while the node assigned sequential IDs from a different pool - the gaps in the sequences between mob kills are due to the fact that other people in the world were killing mobs but all of the items dropped by a single mob were in exact sequential order and the IDs were always in ascending order in the order the mobs were killed.

There is NO way to derive the generic item ID from those unique IDs without looking up the item on the server which is not possible for Lua.

The ExamineItemInstance:ItemInfo values are interesting. They not only contain enough information to identify the item, but also the quantity of a stack as well as what location the item was linked from. Try linking an item from one inventory slot, then move the same item to a different inventory slot and link it - the ExamineItemInstance:ItemInfo value will change but if you move it back to the first slot you will again get the first value. Even more insteresting is that the ID can have a different number of bytes if the item is in bag slots 1,2,or 3 but will have the same number of bytes for all other slots. The same thing seems to occur for a stack if the quantity is 1,2 or 3. The leading part of the ID seems to have two numbers, the first seems to indicate some class of item while the second seems to be the same value for all items, even cross server. Also, the values seem to make more sense if the characters are decoded as Unicode rather than UTF-8.

[Feoktist]

The first item ID in the shortcuts are a Server unique ID for an item or stack of items.
...
There is NO way to derive the generic item ID from those unique IDs without looking up the item on the server which is not possible for Lua.

Thank you Garan! Your analysis is very interesting and disappointing at the same time =(

Just to make clear: I am trying to find a way to derive a short ID for an item from the chat link so that new item information could be entered more easily. Common sense suggests that the item ID should be stored in the ItemInfo binary code. There are probably other pieces of information like slot ID, quantity, quality, creator ID (for craft items) and so forth but this is irrelevant.

I put together a small script to trace all item examination infos flying by and I can relate to what you're saying. I'll do it in a more controlled way today so that some observations can be made.
There seems to be 4 equal encoded characters, "C480C480C480C480", in the beginning of each ItemInfo data. The number of bytes vary from 80 (for vendor loot items like a simple Lute) all the way to 480 (e.g. crafted cloaks and jewellery with creator name and wear state).

Also, the values seem to make more sense if the characters are decoded as Unicode rather than UTF-8.

This is a bit confusing because UTF-8 is an encoding and Unicode is a character set. You mean looking at symbols makes better picture? That's what I started from, but then had to convert the strings of bytes into hex because not all characters are displayed by the client (and written into log files for that matter).

[Garan]

This is a bit confusing because UTF-8 is an encoding and Unicode is a character set. You mean looking at symbols makes better picture? That's what I started from, but then had to convert the strings of bytes into hex because not all characters are displayed by the client (and written into log files for that matter).

Sorry for the confusion. What I meant by using Unicode is to convert the UTF-8 encoded string that Lua provides to a Unicode string and then look at the hex values of the bytes for the Unicode characters. An example is below:

ExamineItemInstance:ItemInfo string:
ĀĀĀĀŜĀĀĀŸƜƛǜũǰƃƉƁƋǹŦĥœāăǓőġĆ āĐŦŦŠŠǰăĒƌČČĎĊżČĂƌƼƎČƩČŅŀƜǃŐ ljƐćƄĊČǑČŮČťČƱĠĵŀŐčŔĥāƤĕřĘĄŀĘŤĞ ĀǻŔĎŻ
UTF-8 bytes:
C4 80 C4 80 C4 80 C4 80 C5 9C C4 80 C4 80 C4 80 C5 B8 C6 9C C6 9B C7 9C C5 A9 C7 B0 C6 83 C6 89 C6 81 C6 8B C7 B9 C5 A6 C4 A5 C5 93 C4 81 C4 83 C7 93 C5 91 C4 A1 C4 86 C4 81 C4 90 C5 A6 C5 A6 C5 A0 C5 A0 C7 B0 C4 83 C4 92 C6 8C C4 8C C4 8C C4 8E C4 8A C5 BC C4 8C C4 82 C6 8C C6 BC C6 8E C4 8C C6 A9 C4 8C C5 85 C5 80 C6 9C C7 83 C5 90 C7 89 C6 90 C4 87 C6 84 C4 8A C4 8C C7 91 C4 8C C5 AE C4 8C C5 A5 C4 8C C6 B1 C4 A0 C4 B5 C5 80 C5 90 C4 8D C5 94 C4 A5 C4 81 C6 A4 C4 95 C5 99 C4 98 C4 84 C5 80 C4 98 C5 A4 C4 9E C4 80 C7 BB C5 94 C4 8E C5 BB

Unicode bytes:
0100 0100 0100 0100 015C 0100 0100 0100 0178 019C 019B 01DC 0169 01F0 0183 0189 0181 018B 01F9 0166 0125 0153 0101 0103 01D3 0151 0121 0106 0101 0110 0166 0166 0160 0160 01F0 0103 0112 018C 010C 010C 010E 010A 017C 010C 0102 018C 01BC 018E 010C 01A9 010C 0145 0140 019C 01C3 0150 01C9 0190 0107 0184 010A 010C 01D1 010C 016E 010C 0165 010C 01B1 0120 0135 0140 0150 010D 0154 0125 0101 01A4 0115 0159 0118 0104 0140 0118 0164 011E 0100 01FB 0154 010E 017B

If you strip off the unicode upper byte which is always 01 for these characters, it seems to present the data more clearly, especially if you are looking for patterns in multiple samples.
00 00 00 00 5C 00 00 00 78 9C 9B DC 69 F0 83 89 81 8B F9 66 25 53 01 03 D3 51 21 06 01 10 66 66 60 60 F0 03 12 8C 0C 0C 0E 0A 7C 0C 02 8C BC 8E 0C A9 0C 45 40 9C C3 50 C9 90 07 84 0A 0C D1 0C 6E 0C 65 0C B1 20 35 40 50 0D 54 25 01 A4 15 59 18 04 40 18 64 1E 00 FB 54 0E 7B

For instance the first five characters seem to represent a number, in this case 0000005c which changes with item type/category (but doesn't line up with the categor from the Lua iteminfo). The next sequence 000000789c seems to be the same for every item even cross server. The next part seems to be the item identifier followed by some descriptors which indicate what additional fields are included and the data for those fields. I would start examining this by capturing as many examine ids as possible, starting from the loot window, then the acquisition message and then as many samples of links from various locations, including equipped locations, as well as viewing the link from both the sender and reciever if possible (requires two computers). If you isolate the portion that stays the same in all of those cases and THEN do the same for another instance of the same item, you should be able to determine what portion contains the identifier. Unfortunately, I have a feeling the identifier will be a Server unique ID which is of very limited use, BUT, since the other data seems to be encoded using that unique ID, you may be able to derive a decoding algorithm which would allow you to see what data is in those additional fields and there may be something useful there.

[Feoktist]

Sorry for the confusion. What I meant by using Unicode is to convert the UTF-8 encoded string that Lua provides to a Unicode string and then look at the hex values of the bytes for the Unicode characters.

Just wanted to say that you've obviously put much thinking into this already, and I really appreciate that you don't mind sharing. My Unicode experience wasn't related to direct encoding / decoding thus I had to digest it first but hey, good news is we're seeing the same strings (I wasn't sure whether RU and global treat strings the same). Converting utf-8 into Unicode symbols and dropping the 1st byte (0x01) yields the same results for me.


My feeling as well is that the byte string should contain some ID and use it to encrypt the string. For instance, Rohan Celebrant Salve looks like this:
[00 00 00 00] [36* 00 00 00] [78 9C] [XX YY**] [26 bytes?] [2 bytes position] [6 bytes?] [2 bytes quantity?] [10 bytes] [4 bytes quantity + position***]
* - Item category: when linked from the vault this byte is 2A and the string is shorter (36 is not unique, e.g. Lothlorien Medallions show 42 when linked from the bag and 36 from the vault.) Limited tests show that green items share this value.
** - Stack ID: 2 bytes seem to be unique for a stack of items i.e. when I move it around it doesn't change. Some samples vs. long ID are
5B E8 = 0x03AB0002AD094FA1
8B 5A = 0x03AB0002C477A25A
BB B1 = 0x03AB0002C474B6D8
7B 78 = 0x03AB0002C477C3E1
It changes when a stack is put into vault and back, or merged into another stack, but not when it's split. Limited tests show that all vault items have bytes "63 60" in this place and hence it's obvious if the item was linked from the vault. This might as well be just "salt" for hashing. Also, it's just a guess that this salt is 2 bytes. The rest of the line is encrypted.
*** - quantity and position parts are relevant for decoding the rest, some algorithm combining bitwise shifting, xoring and reversing is used here. The last 4 bytes' values change when EITHER quantity and position changes. Vault items don't seem to have positional component.

[Feoktist]

Limited tests show that all vault items have bytes "63 60" in this place and hence it's obvious if the item was linked from the vault.

This seems to hold for all items not in direct posession of a player. So the only thing you can say is whether a person linking the item has it in the bag or not.

some algorithm combining bitwise shifting, xoring and reversing is used here

There are large portions of the code which seem to be just placeholders. For instance, there is a group of 16 bytes immediately following the ID information which can look like this for green loot (ore, Rohan emblems, scholar items etc. alike):

0A 18 98 8E 0A 31 08 80 30 13 03 03 83 1F 33 03
05 0C 4C 47 85 18 04 40 98 89 81 81 C1 8F 99 81
02 06 A6 A3 42 0C 02 20 CC C4 C0 C0 E0 C7 CC C0
01 03 D3 51 21 06 01 10 66 62 60 60 F0 63 66 60
80 81 E9 A8 10 83 00 08 33 31 30 30 F8 31 33 30
(I didn't capture the remaining 3 but maybe that's it)

All of the variations are equivalent w.r.t. the bitwise shift.
Also of interest, items stacks of 2 seem to have shorter sequences (3 bytes shorter). Might be some error on my side.

[Garan]

Also of interest, items stacks of 2 seem to have shorter sequences (3 bytes shorter). Might be some error on my side.

Not a mistake on your end, as noted in my first post in this thread, stacks with 1,2 or 3 can have a different number of bytes as can stacks that are in bag slots 1,2 and 3. It may actually only be 2 and 3 that are different, I'd have to go back to my notes for that. I'm not sure why 2 and 3 would be handled differently but they seem to be.

[Feoktist]

Not a mistake on your end, as noted in my first post in this thread, stacks with 1,2 or 3 can have a different number of bytes as can stacks that are in bag slots 1,2 and 3. It may actually only be 2 and 3 that are different, I'd have to go back to my notes for that. I'm not sure why 2 and 3 would be handled differently but they seem to be.

The utf-8 line itself is shorter for these quantities (I only came across a stack of 2 being shorter), maybe there are 1-byte utf-8 characters? Although I have a check that should rise a flag if it meets one in the code, and I saw none.

I'm looking at several links and these bytes seem to disappear from the part that I previously identified as the "quantity" part. Only the bytes showing different values are highlighted.

00 00 00 00 36 00 00 00 78 9C 8B 52 9E 77 84 89 61 35 73 C4 5B A6 02 06 A6 A3 42 0C 02 20 CC C4 C0 C0 E0 C7 CC C0 60 C7 C0 E0 50 CD 07 E1 2B B2 30 08 80 30 48 25 00 3B FE 09 A4 (2xTarnished Celebrimbor Symbols)

00 00 00 00 36 00 00 00 78 9C D3 56 99 77 84 89 61 35 73 C4 5B A6 02 06 A6 A3 42 0C 02 20 CC C4 C0 C0 E0 C7 CC C0 60 C7 C0 E0 50 CD C7 20 C0 08 E4 2B B2 30 08 80 30 48 25 00 32 39 09 75 (Tarnished Celebrimbor Symbol)

[Equendil]

Sorry for the confusion. What I meant by using Unicode is to convert the UTF-8 encoded string that Lua provides to a Unicode string and then look at the hex values of the bytes for the Unicode characters. An example is below:

ExamineItemInstance:ItemInfo string:
ĀĀĀĀŜĀĀĀŸƜƛǜũǰƃƉƁƋǹŦĥœāăǓőġĆ āĐŦŦŠŠǰăĒƌČČĎĊżČĂƌƼƎČƩČŅŀƜǃŐ ljƐćƄĊČǑČŮČťČƱĠĵŀŐčŔĥāƤĕřĘĄŀĘŤĞ ĀǻŔĎŻ
UTF-8 bytes:
C4 80 C4 80 C4 80 C4 80 C5 9C C4 80 C4 80 C4 80 C5 B8 C6 9C C6 9B C7 9C C5 A9 C7 B0 C6 83 C6 89 C6 81 C6 8B C7 B9 C5 A6 C4 A5 C5 93 C4 81 C4 83 C7 93 C5 91 C4 A1 C4 86 C4 81 C4 90 C5 A6 C5 A6 C5 A0 C5 A0 C7 B0 C4 83 C4 92 C6 8C C4 8C C4 8C C4 8E C4 8A C5 BC C4 8C C4 82 C6 8C C6 BC C6 8E C4 8C C6 A9 C4 8C C5 85 C5 80 C6 9C C7 83 C5 90 C7 89 C6 90 C4 87 C6 84 C4 8A C4 8C C7 91 C4 8C C5 AE C4 8C C5 A5 C4 8C C6 B1 C4 A0 C4 B5 C5 80 C5 90 C4 8D C5 94 C4 A5 C4 81 C6 A4 C4 95 C5 99 C4 98 C4 84 C5 80 C4 98 C5 A4 C4 9E C4 80 C7 BB C5 94 C4 8E C5 BB

Unicode bytes:
0100 0100 0100 0100 015C 0100 0100 0100 0178 019C 019B 01DC 0169 01F0 0183 0189 0181 018B 01F9 0166 0125 0153 0101 0103 01D3 0151 0121 0106 0101 0110 0166 0166 0160 0160 01F0 0103 0112 018C 010C 010C 010E 010A 017C 010C 0102 018C 01BC 018E 010C 01A9 010C 0145 0140 019C 01C3 0150 01C9 0190 0107 0184 010A 010C 01D1 010C 016E 010C 0165 010C 01B1 0120 0135 0140 0150 010D 0154 0125 0101 01A4 0115 0159 0118 0104 0140 0118 0164 011E 0100 01FB 0154 010E 017B

If you strip off the unicode upper byte which is always 01 for these characters, it seems to present the data more clearly, especially if you are looking for patterns in multiple samples.
00 00 00 00 5C 00 00 00 78 9C 9B DC 69 F0 83 89 81 8B F9 66 25 53 01 03 D3 51 21 06 01 10 66 66 60 60 F0 03 12 8C 0C 0C 0E 0A 7C 0C 02 8C BC 8E 0C A9 0C 45 40 9C C3 50 C9 90 07 84 0A 0C D1 0C 6E 0C 65 0C B1 20 35 40 50 0D 54 25 01 A4 15 59 18 04 40 18 64 1E 00 FB 54 0E 7B

For instance the first five characters seem to represent a number, in this case 0000005c which changes with item type/category (but doesn't line up with the categor from the Lua iteminfo). The next sequence 000000789c seems to be the same for every item even cross server. The next part seems to be the item identifier followed by some descriptors which indicate what additional fields are included and the data for those fields.

Nice finds Garan and Feoktist.

I scratched my head a bit on this tonight (toying with ExamineIA:IAInfo: in my case). Coming back and forth to the idea that the resulting data might still be encoded or more specifically, compressed. I liked the idea, it could explain the lack of a discernible pattern in the data beyond the first few bytes (not even small bytes or ASCII sequences you might expect in regular data). Those last four bytes would make a good candidate for a checksum also.

I eventually decided that the second longword corrected for endianness (for instance 0x0000005C above) looked suspiciously like some sort of size-of-data-to-follow field you can expect in a header, and since that didn't match the actual size of the data, maybe the size of inflated data. I then checked the following bytes against the RFC of the most obvious candidate for a compression scheme: zlib, and lo and behold, 0x789C matches a perfectly standard zlib stream header (32k window, 'deflate' scheme, no preset dictionary, correct 'check bits' and all). Cherry on the cake: zlib streams are in fact terminated with a checksum (Adler-32).

I haven't actually put the idea to test, I'll do that another day, but I'll be damned if it's not correct.

[Fredelas]

I eventually decided that the second longword corrected for endianness (for instance 0x0000005C above) looked suspiciously like some sort of size-of-data-to-follow field you can expect in a header, and since that didn't match the actual size of the data, maybe the size of deflated data. I then checked the following bytes against the RFC of the most obvious candidate for a compression scheme: zlip, and lo and behold, 0x789C matches a perfectly standard zlib stream header (32k window, 'deflate' scheme, no preset dictionary, correct 'check bits' and all). Cherry on the cake: zlib streams are in fact terminated with a checksum (Adler-32).

This makes intuitive sense, since zlib DEFLATE is a compression algorithm used elsewhere in LOTRO. Good eye!

There are a few pure-Lua implementations of this algorithm out there that authors might borrow from to decompress these streams.

[Equendil]

I can now confirm that my previous post was correct.

Example on an unidentified 2nd age minstrel sword, level 65, from the backpack, on a level 85 minstrel:

Raw chat:
<ExamineIA:IAInfo:ĀĀĀĀƇĀĀĀŸƜƳĮźǏǀǀĠDŽǼǬĄţāŃƩLJĒǞǭ őƌƪƌŀġĐdžĎǬƀŸǍŻdžƂǷĂČĂČnjŇƅĘĄŀƘą ĨǪLJnjǀŠDžǀǠŰƈƋŁƀűƹąƐĸĈŤƵųƁŴǽǧŐŤ šĐĀšƐŕǓĵĘĄŀǘĚŪĽĀǔōĖƉ>Minstrel's Sword of the Second Age<\ExamineIA>

UTF8 data in bold decoded using the \u01xx Unicode code page as per Garan's post:

00000000 87000000 789CB32E 7ACFC0C0 20C4FCEC 04630143 A9C712DE ED518CAA 8C402110 C60EEC80 78CD7BC6 82F7020C 020CCC47 85180440 980528EA C7CCC060 C5C0E070 888B4180 71B90590 380864B5 738174FD E7506461 10006190 55D33518 0440D81A 6A3D00D4 4D1689

Binary section in bold inflated through zlib:

3B72EF00 00001203 E6C80170 007548A4 0DB75A01 25010000 00010000 00000000 00000000 00000000 00000000 00000000 0000003E 000000AC EF0170EF 10001000 03C51200 10C51200 10040000 004E0300 003A0000 40C20A00 1001A738 001001C1 0A001087 0A000000 00FF0821 04001021 040010E6 C8017097 28001097 2800103B 72EF0000 001203

135 = 0x00000087 bytes, and looks a lot more like proper data

On to finding out what's what ...

Edit: the generic item ID features prominently as bytes 8-12, big-endian (first 8 bytes as instance ID, it seems). That's for ExamineIA:IAInfo: but I imagine ExamineItemInstance:ItemInfo: will be no different.

[Equendil]

For anyone interested, I uploaded the lua files I'm using to decode chat link data : http://dl.free.fr/i8oQLgpUF

I made use of the lua zlip inflater implemented by David Manura, slightly modified to work for LotRO: https://github.com/davidm/lua-compress-deflatelua

Usage example:

import "deflate";
import "TurbineUTF8Binary";

(...)

Turbine.Chat.Received = function (sender, args)
	local itemData = string.match( args.Message, "<ExamineIA:IAInfo:(.*)>%b[]<\\ExamineIA>" );

	local decodedBytes = Equendil.TurbineUTF8Binary.Decode( itemData );
	
	inflatedData = {};
	local outputFun = function ( byte )
		table.insert( inflatedData, byte );
	end
	
	local deflatedDataAsString = string.char( unpack( decodedBytes ) );
	deflatedDataAsString = string.sub( deflatedDataAsString, - ( string.len( deflatedDataAsString ) - 8 ) );
	Equendil.ZlibDeflate.inflate_zlib( { input = deflatedDataAsString, output = outputFun, disable_crc=true });

	... resulting data in inflatedData as an array of bytes
	... do whatever with it
	
	-- eg
	local humanLegibleHexCode = Equendil.TurbineUTF8Binary.HexString( inflatedData ) );
	Turbine.Shell.WriteLine( humanLegibleHexCode );
end

Note: download link is temporary and will die out eventually, I'll upload something proper on lotro-interface at some point.

[Garan]

For anyone interested, I uploaded the lua files I'm using to decode chat link data : http://dl.free.fr/i8oQLgpUF

I made use of the lua zlip inflater implemented by David Manura, slightly modified to work for LotRO: https://github.com/davidm/lua-compress-deflatelua

Usage example:

import "deflate";
import "TurbineUTF8Binary";

(...)

Turbine.Chat.Received = function (sender, args)
    local itemData = string.match( args.Message, "<ExamineIA:IAInfo:(.*)>%b[]<\\ExamineIA>" );

    local decodedBytes = Equendil.TurbineUTF8Binary.Decode( itemData );
    
    inflatedData = {};
    local outputFun = function ( byte )
        table.insert( inflatedData, byte );
    end
    
    local deflatedDataAsString = string.char( unpack( decodedBytes ) );
    deflatedDataAsString = string.sub( deflatedDataAsString, - ( string.len( deflatedDataAsString ) - 8 ) );
    Equendil.ZlibDeflate.inflate_zlib( { input = deflatedDataAsString, output = outputFun, disable_crc=true });

    ... resulting data in inflatedData as an array of bytes
    ... do whatever with it
    
    -- eg
    local humanLegibleHexCode = Equendil.TurbineUTF8Binary.HexString( inflatedData ) );
    Turbine.Shell.WriteLine( humanLegibleHexCode );
end

Note: download link is temporary and will die out eventually, I'll upload something proper on lotro-interface at some point.

Nice. Hopefully I will get some time to play with this a bit. I've been a bit busy with some RL issues but it looks like this could have some interesting uses.

[Crell_1]

Nice. Hopefully I will get some time to play with this a bit. I've been a bit busy with some RL issues but it looks like this could have some interesting uses.

I hope that some of you use this information to help fill in the kinds of gaps we'll have once data.lotro.com and the lorebook are taken from us.

Thanks for your efforts!

[Garan]

I hope that some of you use this information to help fill in the kinds of gaps we'll have once data.lotro.com and the lorebook are taken from us.

Thanks for your efforts!

At the very least, this should help with ItemTracker since it lets me get a generic item ID for unique items which aren't exposed in the regular drag and drop info for some reason. It may have many other uses as well. I haven't mapped out all of the data it exposes but I've found a number of useful fields including qty, location, LI legacies, LI relics, etc. and there's many more to figure out. So many fun puzzles and so little time...

EDIT:
FWIW, I can confirm that it works for ExamineItemInstance:ItemInfo as well as ExamineIA:IAInfo

[Equendil]

At the very least, this should help with ItemTracker since it lets me get a generic item ID for unique items which aren't exposed in the regular drag and drop info for some reason. It may have many other uses as well. I haven't mapped out all of the data it exposes but I've found a number of useful fields including qty, location, LI legacies, LI relics, etc. and there's many more to figure out. So many fun puzzles and so little time...

EDIT:
FWIW, I can confirm that it works for ExamineItemInstance:ItemInfo as well as ExamineIA:IAInfo

I haven't gone very far into deciphering all the info that's put into IAInfo, but here are my findings so far:



First, what I believe is the structure of the data, on an example (Level 85 minstrel first age sword with 3 legacies, bound, self crafted)

C17AEF0000001203 51400370 00 7548 A40D 1E6A 0225 01000000 01000000 000000 03 [696D0170 2F000000 8B660270 2F000000 5E6D0170 2F000000] 01000000 0000 00000000 00000000 00000000 6A000000 B0F20170 {EF100010 0003 [C5120010 C5120010 09000000 [{42100010 6C2DB843} {4E030000 5500 0040} {200E0010 01 0C [45007100750061006E006F00720020 005B00460076005D00] 0100 00000000} {C10A0010 AE26000000001202} {A4310010 6A000000} {5F1D0010 01000000} {69060010 6D000000} {96490010 03000000} {35080010 180D0000}] {21040010 21040010 51400370} {97280010 97280010 C17AEF0000001203}]}

I coloured what I think are data markers in gray. Square brackets denote an array which is preceded by its size (also coloured grey), curly braces denote some sort of structure that seem to always start with a 4 bytes identifier of some sort (possibly related to code, class ID or something from serialization) which for some reason is repeated twice sometimes. Some or all of them seem to be optional depending on the item at hand.

It appears that most if not all the data is in big-endian order.

Let's move on to what the data means. I'll stick to the following colours:
In red: fields I haven't identified and not seen change so far
In pink: fields I have seen change, but the meaning of which is unclear to me
In gray: data markers & array lengths

First: data always found from the beginning to the first 'structure':

C17AEF0000001203 51400370 00 7548 A40D 1E6A 0225 01000000 01000000 000000 03 [696D0170 2F000000 8B660270 2F000000 5E6D0170 2F000000] 01000000 0000 00000000 00000000 00000000 6A000000 B0F20170

The first field (in blue) is quite obvious: it's the instance item ID. It's followed by the generic item ID (in in cyan). Follows a number of fields I'm unclear about, then an array of legacies (3 legacies as per the length preceding it). Each legacy has two fields, an ID (in green, and a field I haven't clearly identified but I believe are flags of some type (in pink. Note that legacy IDs seem to be different for each tier of a legacy.

The first yellow field seems to be related to whether the item is bound. I've only seen a value of 0 if not bound, and 1 if bound

The next two yellow fields, I have unambiguously identified as the number of legendary points left to spent, and the legendary points already spent.

The last yellow field followed by an ID seem to define the LI's default legacy (here: Tactical Damage Rating) and its rank (within all possible ranks such a legacy can take, including lower level LIs etc, hence a high value of 0x6A=106).

(more next posts)

[Equendil]

Following the base item info, we find extra data, as a structure marked with "EF100010", which contains an array (here: 0003 elements). That array in turns contains three sub structures (for which 'markers' repeat twice for whatever reason):

- {C5120010 C5120010 09000000[...] : Another placeholder for more data as an array (of 9 elements here)
- {21040010 21040010 51400370}: The generic item ID (again...)
- {97280010 97280010 C17AEF0000001203}: The instance item ID (again...)


Finally, the middle part, the larger array (C5120010) can contain a variety of other info, so far what I've spotted:

- {4E030000 5500 0040} This is storage info. The yellow field is unmistakably the slot in bags in which an item is stored (goes to 0 in vaults), the second field I'm less sure about, so far I've seen it take three values (corrected for endianness): 0x0000 in vault or shared storage: 0x4000 in bags, 0x2001 when equipped (main hand weapon)
- {200E0010 01 0C [45007100750061006E006F00720020 005B00460076005D00] 0100 00000000}. Found on crafted items, the long hexadecimal chain is just an array of characters (two bytes encoding), which here just translates as "Equanor [Fv]", Equanor being my character who crafted that particular item, not sure what "[Fv]" stands for.
- {C10A0010 AE26000000001202} Appear to be the "bound to" info. The 8 bytes field is a player ID (can be a placeholder or possibly unintialized memory if item is not bound).
- {5F1D0010 01000000} LI level (here: 1)
- {A4310010 6A000000} Repeats the 'rank' of the default legacy (Tactical Damage Rating)


More structures found in that array, but not in this example:

- {A7380010 00} Not sure what that one is about, I've seen the one byte value go from 01 on a bound to account item to 00 when bound to self
- {BC260010 03000000 [{BD260010 3BF40170} {BD260010 776D0170} {BD260010 586D0170]} An array of the legacies on the item, each element of the array is another structure, the only info in each being a legacy ID of the legacies on the item. Appeared on an item after identification at the forge master, disappeared when the item was bound (equipped).

Some other structures (by their 'marker') that I've seen on items but not deciphered at all yet:

- 69060010, 35080010, C20A0010, 7B0E0010, 42100010, 96490010

[Fredelas]

Is Equanor a female character by chance?

LOTRO's string building parser relies on tags within square brackets to indicate grammatical properties of certain words or phrases.

The [Fv] tag represents:

• a feminine name [F] (capitalized, required for declension and verb agreement in some languages)
• a singular noun starting with a vowel sound [v] (required for a/an distinction in English and l' in French).

LOTRO often gets these things wrong in its own data. For example, you've probably defeated "the Bugud" a few times.

I would also expect the legendary item's craft-crit name and reforge-name to be part of the data structure somewhere as strings.

[Equendil]

Is Equanor a female character by chance?

LOTRO's string building parser relies on tags within square brackets to indicate grammatical properties of certain words or phrases.

The [Fv] tag represents:

• a feminine name [F] (capitalized, required for declension and verb agreement in some languages)
• a singular noun starting with a vowel sound [v] (required for a/an distinction in English and l' in French).

LOTRO often gets these things wrong in its own data. For example, you've probably defeated "the Bugud" a few times.

I would also expect the legendary item's craft-crit name and reforge-name to be part of the data structure somewhere as strings.

Ahh, I wouldn't have guessed that one. Yes, Equanor is a female character indeed

I indeed expect to see those other strings on other items, I've only toyed with fairly basic LIs so far, mostly trying to decipher the structure of the data more than the content thereof, should be easier now.

[Equendil]

Here's an update regarding data found in item links, for anyone still interested

First, on all items, whether LIs or not, we find (in order):
- 8 bytes: instance item ID
- 4 bytes: generic item ID

If the item is a LI:
- 1 byte indicating whether the item has a custom name (value = 1 if custom name, 0 otherwise)
- if item has a custom name:
	- 1 byte indicating the number of characters in the name
	- the name of the item as a sequence of characters in UTF-16 (2 bytes per character)
- if item does not have a custom name:
	- 2*4 bytes, I assume at least one ID to a localizable resource
- 4 bytes: unknown meaning, never seen anything but a value of 1
- 2 bytes: unknown meaning, never seen anything but a value of 1
- 4 bytes: the ID to a LI title
- 1 byte: unknown/unused
- 1 byte: number of legacies
- n legacies, for each:
	- 4 bytes: ID of the legacy
	- 4 bytes: rank of the legacy
- 1 byte: unknown/unused
- 1 byte: number of relics
- n relics, for each:
	- 4 bytes: ID the relic
	- 4 bytes: type of the relic (value of 1 to 4 for gem/setting/rune/crafted, didn't write down which is which)
- 4 bytes: number of additional stats (stats of 2h LIs)
- n * 4 bytes: IDs to stats
- 4 bytes: legendary points spent
- 4 bytes: legendary points left
- 4 bytes: DPS rank if a weapon, or 0
- 4 bytes: Rank of the primary LI legacy (Tactical Damage Rating, Tactical Healing Rating, Shield Use, etc) or 0 if a weapon
- 4 bytes: ID to the primary LI legacy or 0 if a weapon
- 4 bytes: 0x100010EF always

On both regular items and LIs:
- 1 byte: unknown/unused
- 1 byte: number of element in the top level array as described in my previous posts
(...)


Possible fields in the secondary array and their meaning:
- 0x10000E20 : "Crafted by" data, followed with:
	- 1 byte: unknown/unused
	- 1 byte: number of characters in the name of crafter
	- the name of the item as a sequence of characters in UTF-16 (2 bytes per character)
	- 4 bytes: unknown, always seem to be the same (0 ? 1 ?)
	- 2 bytes: unknown, always seem to be the same (0 ? 1 ?)
- 0x10000884 : Crafting name of the item, same structure as above
- 0x10000AC1 + 8 bytes : "bound" ID of the player to which the item is bound, or a special ID for bound to account
- 0x10001D5F + 4 bytes : Level of the LI (value between 1 - 70)
- 0x100031A4 + 4 bytes : rank of the primary LI legacy (redundant with prior info)
- 0x100026BC: list of legacies (on unbound identified items, apparently)
	- 4 bytes: number of legacies
	- n * 2*4 bytes: 0x100026BD + ID of legacy (redundant with prior info)
- 0x100000C4 + 4 bytes : item level as displayed to players (eg: 85)
- 0x10000669 + 4 bytes : the 'real' item level (currently up to 110 or so)
- 0x10004996 + 4 bytes : number of upgrades (crystals)
- 0x1000132C + 4 bytes : durability (as in: current value, not the max)
- 0x10000835 + 4 bytes : worth of an item in coppers
- 0x10000E7B + 4 bytes : quantity (for stacks)
- 0x0000034E + 4 bytes : storage info (haven't really paid much attention to specifics, possible to identify bag slots)
- 0x10001042 + 4 bytes : second value of dps range in floating point (IEEE 754). First value can seemingly be computed as 3/5 of that (and of course, average hit = 4/5, so DPS = value * 4/5 / weapon swing delay)
- 0x10000ACD + 4 bytes : unknown
- 0x100038A7 + 1 byte : unknown, always seems 0 
- 0x10000AC2 + 1 byte : unknown, always seems 0

Right, that was indigest

Note: while LI links tend to hold a lot of that info, regular item links typically only hold two or maybe three of those fields.