List of item IDs for use with data.lotro.com

[Duwis]

As some of you may have noticed (and PM'ed me), the recent Lorebook update altered the format of the AJAX response for the item advanced search. Specifically, the "rel" attribute containing the item ID. As this was one of the more feasible ways to scrape a list of active items, this presented a problem for the developers using data.lotro.com.

And many of these developers have long clamored for data.lotro.com to have an API call for getting such a list. Well, I have semi-good news; data.lotro.com does not have the API call yet, but as a temporary solution, I have generated an XML file containing the list. You can find said list here:
http://content.turbine.com/sites/lor..._item_list.xml


Again, this is a temporary solution; long term, we want data.lotro.com to be your one stop shopping plaza for game data.

[Harlinator]

As some of you may have noticed (and PM'ed me), the recent Lorebook update altered the format of the AJAX response for the item advanced search. Specifically, the "rel" attribute containing the item ID. As this was one of the more feasible ways to scrape a list of active items, this presented a problem for the developers using data.lotro.com.

And many of these developers have long clamored for data.lotro.com to have an API call for getting such a list. Well, I have semi-good news; data.lotro.com does not have the API call yet, but as a temporary solution, I have generated an XML file containing the list. You can find said list here:
http://content.turbine.com/sites/lor..._item_list.xml


Again, this is a temporary solution; long term, we want data.lotro.com to be your one stop shopping plaza for game data.

Thanks, Duwis!!!!! This beats the nasty data scraper I was running....it was going to finish around Christmas...D'OH!

Harl

[Entenwyn]

Great news. Thanks Duwis!

[stingray69]

2 things

1) Why oh why didn't you give us some kind of pre warning about this? You must have known this change was gunna happen for a long time and yet all we get is a "ooh, some of you have already noticed you're now screwed".

2) Im aware the 'json query' route was unofficial but you knew full well it was the only way we had to query for items. Why on earth didn't you put an a proper solution in place BEFORE you hobbled the only option we had? Its great to see a step towards getting item lists from data.lotro.com but the temp solution you've listed will force us to request all 43,000 items and then parse them to see if they're the ones we need...

[Duwis]

2 things

1) Why oh why didn't you give us some kind of pre warning about this? You must have known this change was gunna happen for a long time and yet all we get is a "ooh, some of you have already noticed you're now screwed".

mea culpa; long story short, we reworked the template system behind the scenes, and the rel attribute got lost in the shuffle. It was not a planned event to remove it. Once the community brought it to our attention, we wanted to get something in place to compensate in the short term.

2) Im aware the 'json query' route was unofficial but you knew full well it was the only way we had to query for items. Why on earth didn't you put an a proper solution in place BEFORE you hobbled the only option we had?

See above.

[legotho]

When retrieving by id, via the data.lotro.com api, there are 25 items which consistently return:

<?xml version="1.0" encoding="UTF-8"?>
<apiresponse>
 <error message="Unable to retrieve item data." code="ItemFetchError"/><cache_info cached_until_gmt="2011-10-03 01:17"/></apiresponse>

These are:

<item id="1879216034"><![CDATA[Supplies]]></item>
<item id="1879212376"><![CDATA[Disturbed Earth]]></item>
<item id="1879215149"><![CDATA[Blood-soaked Compass]]></item>
<item id="1879220626"><![CDATA[Keystone]]></item>
<item id="1879215151"><![CDATA[Blood-soaked Journal]]></item>
<item id="1879216035"><![CDATA[Supplies]]></item>
<item id="1879212377"><![CDATA[Remains]]></item>
<item id="1879216036"><![CDATA[Supplies]]></item>
<item id="1879215153"><![CDATA[Cage Door]]></item>
<item id="1879215155"><![CDATA[Supplies]]></item>
<item id="1879212371"><![CDATA[Lever]]></item>
<item id="1879215152"><![CDATA[Supply Stores]]></item>
<item id="1879215274"><![CDATA[Supply Drops]]></item>
<item id="1879212375"><![CDATA[Dievlig Banner]]></item>
<item id="1879212658"><![CDATA[White Hand Banner]]></item>
<item id="1879212372"><![CDATA[Keystone]]></item>
<item id="1879212370"><![CDATA[Cairn-entrance]]></item>
<item id="1879212374"><![CDATA[Disturbed Cairn Entrance]]></item>
<item id="1879215156"><![CDATA[Blood-soaked Boots]]></item>
<item id="1879215157"><![CDATA[Dunlending Hut]]></item>
<item id="1879215150"><![CDATA[Banner]]></item>
<item id="1879212369"><![CDATA[Dievlig Standard]]></item>
<item id="1879212373"><![CDATA[Food Stores]]></item>
<item id="1879215154"><![CDATA[Weakened Cavern Ceiling]]></item>
<item id="1879214879"><![CDATA[Calenard Lure Trap]]></item>

There are four which consistently return:

<?xml version="1.0" encoding="UTF-8"?>
<apiresponse>
 <error message="Unknown error encountered." code="unknownError"/></apiresponse>

These are:

<item id="1879188737"><![CDATA[Thatcher's Tunic & Pants]]></item>
<item id="1879188714"><![CDATA[Well-worn Tunic & Pants]]></item>
<item id="1879193210"><![CDATA[Grant Stop, Drop, & Roll Emote]]></item>
<item id="1879102019"><![CDATA[Wheelbarrow & Sacks]]></item>

(inconsistently, it's possible to get the same messages on the other items, but eventually, they do succeed.)

[stingray69]

mea culpa; long story short, we reworked the template system behind the scenes, and the rel attribute got lost in the shuffle. It was not a planned event to remove it. Once the community brought it to our attention, we wanted to get something in place to compensate in the short term.

Ah, I see.

Damn, now I feel like a jerk for jumping to the wrong conclusion

[XrioT]

Any chance you can you include Item Level in the XML?

Main problem I'm having is this:
1879058235 Padded Gloves of Rallying
1879058234 Padded Gloves of Rallying
1879051024 Padded Gloves of Rallying
1879068623 Padded Gloves of Rallying
1879053860 Padded Gloves of Rallying
1879056541 Padded Gloves of Rallying
1879051025 Padded Gloves of Rallying

If we query the LoreBook, and then have to "Lookup" the ItemID based on the given Name's, we have no way of telling which ItemID corresponds to item a user is actually selecting because so many of the items have the same name (and often only differ in level and stats).

Thanks,
X

P.S. Can we just add the ItemID back to LoreBook response? Please?

[CMI]

itemlist url throws e500 for several days.
This should not happen to a static text file.

What's wrong?

[legotho]

Any chance you can you include Item Level in the XML?

Main problem I'm having is this:
1879058235 Padded Gloves of Rallying
1879058234 Padded Gloves of Rallying
1879051024 Padded Gloves of Rallying
1879068623 Padded Gloves of Rallying
1879053860 Padded Gloves of Rallying
1879056541 Padded Gloves of Rallying
1879051025 Padded Gloves of Rallying

If we query the LoreBook, and then have to "Lookup" the ItemID based on the given Name's, we have no way of telling which ItemID corresponds to item a user is actually selecting because so many of the items have the same name (and often only differ in level and stats).

Thanks,
X

P.S. Can we just add the ItemID back to LoreBook response? Please?

I'm not sure I understand why there's a problem with this: I've just had a look at the first 3, and they're level 42, 42, and 33 respectively. I'm assuming that's a representative sample But that's using the itemID to look up the details, via data.lotro.com, not the name.

[XrioT]

I'm not sure I understand why there's a problem with this: I've just had a look at the first 3, and they're level 42, 42, and 33 respectively. I'm assuming that's a representative sample But that's using the itemID to look up the details, via data.lotro.com, not the name.

I probably didn't explain well.

The reason we have the XML file is because the lorebook response we were using no longer includes the ItemID. So the proposed solution is to use the XML file "Item Name" to look up the ItemID, right?

If multiple ItemID's can have the exact same name how can we look up the unique ItemID by name?

For example if I search the lorebook for an item and get 5 item results with the same name, when a user clicks one of those items to "Open" the item details screen, how can I get the ItemID of that specific item? I can't use this XML to look up by name because they might click the third item in the list (level 33) and if the look up by name gives the first ItemID it finds for that name, it can end up being the wrong item (level 42).

Make sense?

X

[legotho]

I probably didn't explain well.

The reason we have the XML file is because the lorebook response we were using no longer includes the ItemID. So the proposed solution is to use the XML file "Item Name" to look up the ItemID, right?

Ah. Ok, that's why I'm confused: that's not what I'm doing. I'm using it a complete catalogue of IDs, and don't really care what name is in the XML file. I've actually used it to pull all 40000-odd items out of data.lotro.com, and I'm using those XML data files, rather than the lorebook. So I'm *not* trying to work backwards from the lorebook.

If multiple ItemID's can have the exact same name how can we look up the unique ItemID by name?

For example if I search the lorebook for an item and get 5 item results with the same name, when a user clicks one of those items to "Open" the item details screen, how can I get the ItemID of that specific item? I can't use this XML to look up by name because they might click the third item in the list (level 33) and if the look up by name gives the first ItemID it finds for that name, it can end up being the wrong item (level 42).

Make sense?

X

Yeeeees, I understand that that's not going to work.

I'm not quite sure what you're trying to do though. If a user searches in the lorebook, and then clicks one of the multiple "Padded Gloves of Rallying", I'm not sure how (or why!) you're even capturing that action. I guess you're embedding web pages into some kind of hosting application? (If I knew what your application was, I'd be in a better position for thinking about it. As it is, the rest of this is pretty much guesswork...)

But what are you doing with the ItemID? You could just use the HTML for the linked page (and you've got a unique URL for that page, if you need to recall it later). If you don't need to extract the item's properties, but just want to display it in a pleasing style, I'd suggest that's probably the better way to do it, and the itemID is irrelevant. If you *do* want the item properties for your own processing, you could parse them from the HTML (which is decidedly less than ideal, but it is possible).

However, if you really want the itemID in the first place, so you can use the data.lotro.com data, my approach is to extract the entire database first, and then write my own search interface, completely ignoring the lorebook search. Now I've written that, I can see why you might not want to do so, but I'd think it's better than trying to write some kind of hybrid process which is trying to parse data out of HTML pages which are subject to having the structure changed anyway (as the sudden absence of the itemID demonstrates!)

[XrioT]

Ah. Ok, that's why I'm confused: that's not what I'm doing. I'm using it a complete catalogue of IDs, and don't really care what name is in the XML file. I've actually used it to pull all 40000-odd items out of data.lotro.com, and I'm using those XML data files, rather than the lorebook. So I'm *not* trying to work backwards from the lorebook.



Yeeeees, I understand that that's not going to work.

I'm not quite sure what you're trying to do though. If a user searches in the lorebook, and then clicks one of the multiple "Padded Gloves of Rallying", I'm not sure how (or why!) you're even capturing that action. I guess you're embedding web pages into some kind of hosting application? (If I knew what your application was, I'd be in a better position for thinking about it. As it is, the rest of this is pretty much guesswork...)

But what are you doing with the ItemID? You could just use the HTML for the linked page (and you've got a unique URL for that page, if you need to recall it later). If you don't need to extract the item's properties, but just want to display it in a pleasing style, I'd suggest that's probably the better way to do it, and the itemID is irrelevant. If you *do* want the item properties for your own processing, you could parse them from the HTML (which is decidedly less than ideal, but it is possible).

However, if you really want the itemID in the first place, so you can use the data.lotro.com data, my approach is to extract the entire database first, and then write my own search interface, completely ignoring the lorebook search. Now I've written that, I can see why you might not want to do so, but I'd think it's better than trying to write some kind of hybrid process which is trying to parse data out of HTML pages which are subject to having the structure changed anyway (as the sudden absence of the itemID demonstrates!)

Thanks for the great feedback! Yes, i'm basically "piggybacking" off of the data to avoid writing my own searching and maintaining my own services or database. It has worked great for other games because suprisingly once these sites are created the query results almost never change (can't speak the same for Lotro obviously since I just released the app to Android Marketplace a month ago and it already changed).

I have a front-end model that provides a simple means to do this (and even makes adjustments dynamically if the data changes) as long as "enough" data is available I'm good to go. Ultimately I'm just trying to avoid re-writing and maintaining all of the same functions which the search sites themselves already do very well (searching and presenting up-to-date data in a timely manner).

I had thought as-well to just hit the API 40,000 times to retrieve all of the data I need for the items but thought that might be excessive and set off a red-flag on the Dev API Key. I'll probably just go that route at this point, especially since it sounds like thats what you've done without issues anyway

Thanks a lot for your time and explaination,

X

[MolleaEU]

Pushing up this thread as I noticed that there are some game items missing from the lorebook and (maybe) from data.lotro.com (I checked the data in the XML file linked in this post).

While checking the data and crossing it with the items I have on my toon, it seems that there are some items missing. I can name some here:
* Heavy lion's cloak
* Trollslayer Gauntlets
* Thoughtful Band
* Dolen-gwobor
* Cruel Iron Loop
* Elf-wrought Iron Chain

I may have mistyped some, but when searching for substrings those don't come out. Most of these items were obtained via RoI quests, drops, or in lvl 75 school/libraries.

Is this an error or by design?

[bitman0101]

Duwis, 2 questions please.

1) Can you please give an update on the new plan for exposing the ability to search for items and obtain their associated item ID. The list you referenced above is link dead.

2) Is there a plan to update the lorebook information (and therefore data.turbine.com) to better reflect the in game content? Specifically it appears that the pocket items from the last chapter of the epic book (Angowir, Dolen-Gwobor, Dreng-Gylp, Hyrde-Gylp) are missing, and perhaps more importantly there is currently no information on the items that drop from the revamped instances (12-Man RAID skirmishes and 3-man instances), nor is there information available for items dropped in the 24-Man Draigoch RAID. Can we expect these to be available in the lorebook and therefore data.turbine.com?

Any updates you have would be greatly appreciated.

Thanks!

[lunarwater]

Does anyone have a copy of the xml dump mentioned here that they can mirror? I'd like to compare it with the item ids I harvested for Compendium and ensure I didn't miss any. Thanks!

[Quijar_EU]

Does anyone have a copy of the xml dump mentioned here that they can mirror? I'd like to compare it with the item ids I harvested for Compendium and ensure I didn't miss any. Thanks!

Hi,

Here: http://files.siliconworld.org/MMORPG..._item_list.xml
or in SQL format: http://files.siliconworld.org/MMORPG...A/lb_items.sql

Regards.

[lunarwater]

Hi,

Here: http://files.siliconworld.org/MMORPG..._item_list.xml
or in SQL format: http://files.siliconworld.org/MMORPG...A/lb_items.sql

Regards.

Thank you!

[bitman0101]

The purpose of this post is to close the loop on this topic as we haven't heard any news lately.
I've been informed by Turbine that the new policy to handle missing item information is to open a bug report for each item of interest.

Further I've been told that we should categorize the bug using the following options:
LOTRO Community/Forum Support =>My.LOTRO=>"Blog & Other Inquiries"

As we will be filing a bug report, we won't get a response from customer support like we do for tickets. We'll just have to hope that these bug reports make it to the appropriate personnel and receive some attention.

I'm not sure how efficient this process is given the large number of items that are currently missing from the lorebook (most (all?) of the new teal item drops from Raid skirmishes, classic instances, the Draigoch raid, and some epic line), but it is the only official support method available to us at this time.

Updates and missing items aside, I eagerly await news of an official process (or even a workaround) for obtaining new item IDs that we can use to query information from the data.turbine.com infrastructure.

With the removal of the static item list referenced above, and without the "rel" tags previously available, we have no way of knowing what Item IDs are available now, let alone all of the items and associated item IDs that are to come with Update 5 looming.

The data.turbine.com tool is a great tool with a lot of potential. Unfortunately, it has been largely rendered ineffective until this problem is solved.

[Gondoron]

Is this file still there? 500 error trying to fetch it.

[bitman0101]

No. Turbine appear to have pulled it.

[Harlinator]

Duwis,

Any way we can get a new list of item IDs containing the U4/U5 items recently added to Lorebook?

Thanks,
Harl

[TheFrank]

Duwis,

Any way we can get a new list of item IDs containing the U4/U5 items recently added to Lorebook?

Thanks,
Harl

Send me a PM or game mail. I have something even better.

EDIT: nm IID would be the ingame IID which is different than the lorebook uses...which I see was just updated too. If you still want it, let me know

[Harlinator]

No biggie at this point. I brute-force found most/all the latest item IDs by simply querying data.lotro.com and tossing the XML that contained error messages. It was just convenient to have this list when RoI came out.

My biggest gripe now is that there's no way to determine the instance where an item drops for many of the U5 items. For some, the Dropped in Dungeon info is on the Lorebook page, but for most of the U5 items, this info is MIA

Harl

[Harlinator]

Here's a link to the list of IDs that I have. It is not guaranteed to be complete, etc, etc.

Let me know if the link doesn't work for you.

http://cdome.guildportal.com/GuildFiles/212240/ids.txt

Harl