Will World Transfers from EU to NA (and vice versa) ever be available?

[UbbaGlor]

Title says it.

A lot of people move countries and they would like to keep playing their character in a time zone that is convenient for them. Will Turbine ever allow that to happen?

[Sapience]

Title says it.

A lot of people move countries and they would like to keep playing their character in a time zone that is convenient for them. Will Turbine ever allow that to happen?

Unfortuantely due to some data issues with the EU worlds that weren't known until we took back the service, this isn't possible. it's not a question of not allowing it, it's a technical issue that prevents us from offering it. We'd love to let people go to whatever server they wished and move from server to server any time they like, but due to this issue we cannot.

[UbbaGlor]

Thank you for your quick reply Sapience

[Kerin_Eldar]

Unfortuantely due to some data issues with the EU worlds that weren't known until we took back the service, this isn't possible. it's not a question of not allowing it, it's a technical issue that prevents us from offering it. We'd love to let people go to whatever server they wished and move from server to server any time they like, but due to this issue we cannot.

Because writing some code to convert numbers from one format (the EU data) to another (the NA data) is hard? Sapience, this is simply a DATA problem, you say it yourself, professional developers spend their working lives manipulating DATA.

At the end of the day an EU character has EXACTLY THE SAME ATTRIBUTES as a corresponding NA character .. meaning an identical character as configured by their stats, gear, etc. .. can exist in both systems so patently the data that expresses the EU character's attributes (ie. character stats, item identifications for gear, inventory etc.) can be transformed into that of the NA equivalent. There is no item or stat available to an EU character that's not available in the NA system, thus there is no DATA that cannot be transformed .. this this is NOT impossible as you assert every time this comes up.

LACK OF WILLINGNESS to develop this DATA TRANSFORMATION tool is the blocker here, not some unspecified "data issues", please at least be honest!

[trcanberra]

Because writing some code to convert numbers from one format (the EU data) to another (the NA data) is hard? Sapience, this is simply a DATA problem, you say it yourself, professional developers spend their working lives manipulating DATA.

At the end of the day an EU character has EXACTLY THE SAME ATTRIBUTES as a corresponding NA character .. meaning an identical character as configured by their stats, gear, etc. .. can exist in both systems so patently the data that expresses the EU character's attributes (ie. character stats, item identifications for gear, inventory etc.) can be transformed into that of the NA equivalent. There is no item or stat available to an EU character that's not available in the NA system, thus there is no DATA that cannot be transformed .. this this is NOT impossible as you assert every time this comes up.

LACK OF WILLINGNESS to develop this DATA TRANSFORMATION tool is the blocker here, not some unspecified "data issues", please at least be honest!

Cost would of course come into it. Depending on demand they have seen I imagine they would only consider it for full cost recovery. Still, it would be interesting to see what it would cost per transfer to justify the time / expense.

[Kortaan]

At the end of the day an EU character has EXACTLY THE SAME ATTRIBUTES as a corresponding NA character

unfortunately, no. EU characters used to be on Codemasters servers, and they basically did not use the same way to describe the characters on EU as Turbine did on NA. In a similar thread, Sapience already stated:

Unfortunately, due to serious issues with the data provided to us from the former EU servers when we took the service back, it's highly unlikely. The short version is there were some data structures and policies in place that, when we took the service back, we discovered had not been followed. As a result, copying characters between the two sets of servers isn't possible without putting some portion of those characters transferred at extreme risk as well as some data on the server that character is being transferred to.

[Yula_the_Mighty]

Because writing some code to convert numbers from one format (the EU data) to another (the NA data) is hard?

Without seeing the EU and NA data format there is no way for us to know how easy or hard it is. In general the hardness is due to:

1) No definition of what the EU data format is. It was defined by Codemasters. It is highly unlikely Codemasters turned over this information.
s) Somebody is going to have to generate the EU data format in a human readable form
b) Somebody is going to have generate the NA data format in a human readable form. I doubt after 7 years there is an up to date document.

Unfortunately the server do not read human. Plus it is an error prone process. You probably have to do it the hard way.

2) Take samples of the EU data. Try to convert them to NA format. Generate a document as you go thru this process. Use the data to make sure it is done correctly. Unfortunately this method requires highly trained employees which are in high demand to do other stuff. You are never really sure you go it right until you live deploy and see how the customers like it.

3) Turbine may not have the human resources - people with the correct skills to do this activity. They have to hire expensive contractors.

If Turbine was planning to make these kind of transfers available they would not do a data conversion as part of the character transfer tooling. They would have changed all the Codemasters servers to NA data format. Getting rid of the second format would permanently reduced their operating cost. The time to have done this change would have been during the run up to taking over the EU servers.

Companies like West Fargo and other big banks do this activity all the time. It takes them 1 - 2 years. Costs millions of dollars to accomplish the task. This kind of cost is beyond anything Turbine or Warner Brothers would be willing to pay. The entire reason for converting Wachovia accounts to Wells Fargo data format is to reduce Wells Fargo's operating because they do not want two banking systems that are linked together via common applications like a web browser.

One final thought. EU data format is going to have common elements with NA data format, elements that are only found in EU data format, missing elements that are only found in NA data format. Part of the problem is defining a new data format that includes all the NA and EU data elements. You have to resolve any conflicts between data elements. You have to figure out how to set all the NA specific data elements and or EU specific data elements.

[Edwar368]

This is a "data migration" project and have to agree with the above poster, they are never as easy as it may seem. Having worked on one of the worlds largest public funded Data Migration project, for over 5 years now, almost anything is possible if you throw (waste?) enough money at it. But I doubt there would ever be enough takers for this option to get even close to cost of the project. Sadly I am a UK player on a NA server, simply because when I started playing last year I had no idea there were EU servers and I just chose the "recommended" server. SO I want this to, but am aware it wont happen and understand why

[Ellieni]

I've always wondered why if we can not have US to EU transfers and vice versa, why a different service can not be offered to cater for players who want to move to another server but do not want to start all over again.

Couldn't you offer a paid for service that gives the player a character of the same sex, level and class on a server of their choice and just delete the character on their old server? Obviously deeds, quests etc would have to be done all over again. Whilst not a perfect solution, I think that would satisfy a lot of people.

[cdq1958]

I've always wondered why if we can not have US to EU transfers and vice versa, why a different service can not be offered to cater for players who want to move to another server but do not want to start all over again.

Couldn't you offer a paid for service that gives the player a character of the same sex, level and class on a server of their choice and just delete the character on their old server? Obviously deeds, quests etc would have to be done all over again. Whilst not a perfect solution, I think that would satisfy a lot of people.

I suspect that this will not be done because the real world result would be worse than what we have now. There is an saying that applies to this situation. The cure is worse than the disease.

[Easca]

While in theory it looks easy, in reality it is nearly impossible. Well, of course, nothing is impossible when you throw enough money at it, but that would either mean Turbine needs a lot of server transfers to outbalance the costs, or it needs to ask for unreasonable amounts of money - neither of which is really expectable.

In the current situation, if you were to transfer characters from an NA to an EU server, what would happen would be data conflicts that would make your character (partially) broken, create bugs and all of that kind of stuff. Maybe a good comparison would be if a monolingual anglophone from the United States of America were to be placed in the completely culturally and linguistically different Denmark: basically, most of what is understood is left, however, you do not speak the local language and you have no idea what people are saying to you in Danish - the shortcomings, the bugs, the problems. It would take time and perhaps even money to iron those shortcomings out, and at this moment I am guessing that the cost and human resources for such an undertaking are not viable enough for Turbine's economical welfare.

[Ellieni]

I suspect that this will not be done because the real world result would be worse than what we have now. There is an saying that applies to this situation. The cure is worse than the disease.

I think you may be right. I suppose if that Gift of the Valar store item ever becomes available it will go some way in helping peeps who for whatever reason need to start again on a new server.

[maartena]

LACK OF WILLINGNESS to develop this DATA TRANSFORMATION tool is the blocker here, not some unspecified "data issues", please at least be honest!

You obviously haven't dealt with major migrations in a large IT environment. I work for a company with over 10.000 servers. (yes servers, not workstations), among which are older systems such as IBM Mainframes and Supercomputers. Those systems have a unique way of storing and transferring data between databases and environments. We are now faced with converting those systems, and not only converting them but doing it in such a way everything works perfectly the same. Our chosen target is RedHat Linux servers, and in some cases Windows servers. Converting the code and data from these older systems costs many months of development, many manhours, and often at a cost that exceeds a 7-figure number. In some cases, conversion is only possible by converting, and redeveloping it onto an "in between" system that is able to then be converted to Linux, such as something that will work on older HP-UX systems which then can be moved to Linux at some stage after. This all, of course, while maintaining 24/7 uptime.

Without knowing the facts, we can't know how much time, money and effort it would take. In the end it will be a simple game of math: They will estimate how many people are willing to pay $25 per toon. Then they estimate how many people are willing to give up their accounts and VIP subscriptions if they do NOT get what they want. And then they will estimate how much it will cost to get the work done.

If in the end, they estimate it will cost MORE money then it will bring in, there is no good business reason to do so.

This all said, how long did it take you to level a toon from 1 to 85 or 95? I have done it in as little as a month, with questing and deeding, but I have heard of extreme cases where they have done it in a week. Hell, I was 95 withing 1 day of Helms Deep release. If you want to move to a US or EU server, start a toon there.

[Sapience]

Because writing some code to convert numbers from one format (the EU data) to another (the NA data) is hard? Sapience, this is simply a DATA problem, you say it yourself, professional developers spend their working lives manipulating DATA.

At the end of the day an EU character has EXACTLY THE SAME ATTRIBUTES as a corresponding NA character .. meaning an identical character as configured by their stats, gear, etc. .. can exist in both systems so patently the data that expresses the EU character's attributes (ie. character stats, item identifications for gear, inventory etc.) can be transformed into that of the NA equivalent. There is no item or stat available to an EU character that's not available in the NA system, thus there is no DATA that cannot be transformed .. this this is NOT impossible as you assert every time this comes up.

LACK OF WILLINGNESS to develop this DATA TRANSFORMATION tool is the blocker here, not some unspecified "data issues", please at least be honest!

Actually, without going into depth here, it's not as simple as you think. I think you're making a huge assumption. This has nothing to do with EU character sets. It has to do with fundamental database architecture and how things were handled. It is not a trivial task. To be more blunt, you are wholly incorrect in your assumptions and as a result completely incorrect in your conclusions.

Given how often we get accused of doing things just to make money, and how often this question comes up (people requesting transfers from EU to NA shards and vice-verse), don't you think we'd be willing to do something that would make us money if it were as simple as you say? In fact, it would be a bit of a no-brainer.

[Tardall]

... It has to do with fundamental database architecture and how things were handled. It is not a trivial task.

In support of this:
a few months ago, the company I work for deployed a new ERP system.
With a database structure which is totally different from the old one and so are many of the data formats.

Migrating the data from the old ERP system to the new one was nothing but a bloody nightmare.

Developing the tools for the data migration took 3 specialists 1 1/2 year (yes the system is kinda big, with all sorts of legal requirements to take into consideration too).
Before deploying the new system we have gone through testing some 7500 different scenarios multiple times.

We are now a few months after the new system has been deployed and we are still finding and fixing data migration errors.
Some of which brought the new system almost to a grinding halt.


So yeah, migrating data bewteen two fundamental database structures is far from trivial.

[Angadan]

..At the end of the day an EU character has EXACTLY THE SAME ATTRIBUTES as a corresponding NA character..

...you're making a huge assumption..has to do with fundamental database architecture...wholly incorrect in your assumptions and as a result completely incorrect in your conclusions....

Sapience says you make a HUGE assumption that is WHOLLY incorrect, Kerin. Reading what you wrote, the biggest possible assumption I can see is what I isolated above from your post. Maybe EU and NA toons are NOT built the same and it only SEEMS that way when we look at our character panels.

Now there's an interesting thought.

In any case, sorry this will never happen. It would be fun to see more NA-EU migration in-game. I'd go to France for a bit.

[Tardall]

Sapience says you make a HUGE assumption that is WHOLLY incorrect, Kerin. Reading what you wrote, the biggest possible assumption I can see is what I isolated above from your post. Maybe EU and NA toons are NOT built the same and it only SEEMS that way when we look at our character panels.

Now there's an interesting thought.

In any case, sorry this will never happen. It would be fun to see more NA-EU migration in-game. I'd go to France for a bit.

If I may:

I think the assumption Sapience was referring too was the unwritten and underlying assumption that the database structure and data formats in use at the EU and NA servers are identical or close to identical.


As I understand it: the reality is quite different.

[Almagnus1]

Actually, without going into depth here, it's not as simple as you think. I think you're making a huge assumption. This has nothing to do with EU character sets. It has to do with fundamental database architecture and how things were handled. It is not a trivial task. To be more blunt, you are wholly incorrect in your assumptions and as a result completely incorrect in your conclusions.

Given how often we get accused of doing things just to make money, and how often this question comes up (people requesting transfers from EU to NA shards and vice-verse), don't you think we'd be willing to do something that would make us money if it were as simple as you say? In fact, it would be a bit of a no-brainer.

So why can NA and EU copy to Bullroarer?

[Brandon_Blackbird]

So why can NA and EU copy to Bullroarer?

Was something changed to allow them to do it at the same time? Last time I participated, they had to flip between NA and EU and wipe the database in between, and if it was EU, NA couldn't transfer and vice versa.

[Almagnus1]

Was something changed to allow them to do it at the same time? Last time I participated, they had to flip between NA and EU and wipe the database in between, and if it was EU, NA couldn't transfer and vice versa.

Yeah...

I'm curious what the differences are.... i mean, there's prolly enough of us that are tech savvy that they could crowd source it, in theory.

[Ernuvorn]

The only way I could see something like this happening is if (and this is a huge if due to cost) Turbine were to manually code the EU/NA character to the NA/EU format from scratch. This would be a huge if not impossible task to perform and would take so long that it is not really a viable option. The cost to each person that wants to do this for each character they wanted to transfer would be astronomical. I can't really see this happening for that reason alone. No one would want to spend (this is a huge guess based on approx. 150 hours of coding and testing per character to rewrite at approx $20/hr for the dev time) $3000 per character (again only an assumption of the time and cost to write the character from scratch) seeing as there would be almost no way to know for sure that the character would not be corrupted in the process. However, as quick and easy as it is to level up now (don't get me started on that) and gear a character up, it would be much easier to just start a new character on the server you want to be on. That would actually be an easy way to do it.

[Almagnus1]

The only way I could see something like this happening is if (and this is a huge if due to cost) Turbine were to manually code the EU/NA character to the NA/EU format from scratch. This would be a huge if not impossible task to perform and would take so long that it is not really a viable option. The cost to each person that wants to do this for each character they wanted to transfer would be astronomical. I can't really see this happening for that reason alone. No one would want to spend (this is a huge guess based on approx. 150 hours of coding and testing per character to rewrite at approx $20/hr for the dev time) $3000 per character (again only an assumption of the time and cost to write the character from scratch) seeing as there would be almost no way to know for sure that the character would not be corrupted in the process. However, as quick and easy as it is to level up now (don't get me started on that) and gear a character up, it would be much easier to just start a new character on the server you want to be on. That would actually be an easy way to do it.

Nah, what you do is code up a translator, then use the Bullroarer server to test it out with the next update. Once you have a translator that works reasonably well, you run it through a small server's database snapshot, and start comparing results with source, and working out errors (as that would be a decent sample size). Once you're happy there, run it through progressively larger pop server snapshots until it works as desired on a snapshot from one of the bigger servers. By that point, it should be so thoroughly tested and worked that it should be essentially safe.

At least, that's the high level of how that would work if I was going to translate the character data.

However, without knowing what the database structures are, where the differences lie, and how much data per character we're really talking about, there's no way to accurately gauge how long it would take to write a rough draft of the translator application. Given what Sapience has said, we're probably dealing with a lot of data per character.

[Neumi]

Impossible = too expensive.

SOMEONE has to pay for it. It will take a lot of dev time to convert the data (and test the system, fix bugs) and unless you don't find several thousand chars to transfer, it will be a loss for Turbine.

[Runesi_EU]

So why can NA and EU copy to Bullroarer?

Because the element of risk that is involved can pretty much be ignored. If you character copy from an EU server to Bullroarer and your character gets corrupted, no big deal. If you do this on live and you lose your character permanently, that is a big deal.

I had a number of occasions in the last beta where I tried to copy a character from Snowbourn to Bullroarer and it failed, I couldn't then attempt to copy the character again until the server wipe for the next round of beta. Had this happened on live my character would have been lost.

[mjk47]

Because the element of risk that is involved can pretty much be ignored. If you character copy from an EU server to Bullroarer and your character gets corrupted, no big deal. If you do this on live and you lose your character permanently, that is a big deal.

I had a number of occasions in the last beta where I tried to copy a character from Snowbourn to Bullroarer and it failed, I couldn't then attempt to copy the character again until the server wipe for the next round of beta. Had this happened on live my character would have been lost.

Precisely. it is a copy, not a move. If it is broken on arrival, it doesn't actually matter.