Dev Diary Feedback: QA: We’re Not Just Playing Around!

[Boraxxe]

QA is very much involved with evaluating difficulty. The answer here is that the first Book of a new Volume is intended to not be overly challenging.

MoL

Thank you. I now understand. I will get my a$$ handed to me later in the series.

[Grhysli]

I think you need to move the Weathertop Torch about 2 feet to the left. :-)

[willwheels]

Do you have a suggestion for a specific team you'd like to know more about?

Team Ramrod, Team Chaos, Team Awesome, and Team Lakehouse.

One of these things is not like the other. What gives?

[SaintBass]

Do you have a suggestion for a specific team you'd like to know more about?

Can only speak for myself, but I would love to hear from those that build the landscape and instances. IMO, they are some of the best in the business.

[Nasty8]

Do you have a suggestion for a specific team you'd like to know more about?

I'd love to hear about the PvMP team or the instance/raid design team.

Other than that, fun article to read... I was a QA tester for EA a few summers back and came out of that summer with new-found respect for the different hands and helpers that go into what makes a game tick. Definitely an eye-opening experience for me.

That being said, I've been impressed with the quality of most of the content that has come out lately... just not necessarily the quantity. Is there a Quantity Assurance team?

[Fredelas]

That being said, I've been impressed with the quality of most of the content that has come out lately... just not necessarily the quantity. Is there a Quantity Assurance team?

I think you're looking for the accounting department. It's that way --->

[Nasty8]

I think you're looking for the accounting department. It's that way --->

Touche... well played.

[Golledhel]

Do you have a suggestion for a specific team you'd like to know more about?

World Builders.

[Tuco]

Thanks for the dev diary. I very much appreciate all the hard work that goes into the quality checking -- even when I come up with some clever idea for getting over a fence or around an obstruction and find that somebody's thought of that and blocked it off!

One question: how do you plan to integrate the formal testing with the slightly-washed and usually unruly masses on Isengard? Will it be like +Hengar's to-do list for Bullroarer before the recent update?

[7ofswords]

I have some questions. What is the situation of the QA team's gear, items, traits, LIs etc. when testing instances? Do all the QA testers run in with maxed everything or not?

The majority of players in-game do not experience these instances in maxed, top tier items so do you actually account for this by testing these instances with subpar virtues, equipment, weapons, etc. as the average player would?

I think this is an excellent place for me to step in.

Hi, I'm 7ofswords and I run the qualitative test team, internally referred to as Team Lakehouse. As Brun touched on we perform legitimate passes on new content to help developers tune difficulty. My team of LOTRO addicts are dedicated to performing nothing but this type of iterative testing. If your mother ever told you that playing video games would never get you anywhere never trust anything she says ever again.

To answer your question we use gear, traits and IA that scale based upon the intended step in gear progression of the content we're testing. We also play with different party compositions.

I hope that succinct PR friendly response answers your question.

[Boraxxe]

To answer your question we use gear, traits and IA that scale based upon the intended step in gear progression of the content we're testing....

So you guys have "adjustable" gear and traits? Sounds interesting.
Any chance that stuff will be available through the Lottery?

[Sonsy]

I had a part-time job doing QA for a small software company once, and yeah...it was interesting and boring at the same time. I basically had a notebook of procedures. Every time there was an update to the program, I went through each one step-by-step to make sure it worked right and report anything odd. I never really got into the ins and outs of exactly what I was looking for...I was mostly there just to make sure it didn't crash or otherwise completely not work on a couple of alternate operating systems.

It was interesting the first few times...when nothing worked right and sometimes produced astonishing results. By about the 10th iteration when everything was working, it got pretty dull. Still, it had to be tested each time in case something that had worked was accidentally broken.

Of course, it was a very different type of program than LotRO. You could easily classify things as either working or not working. In LotRO, I imagine that there are many, many shades of grey.

So, good job QA team...we actually DO appreciate what you do!

[7ofswords]

So you guys have "adjustable" gear and traits? Sounds interesting.
Any chance that stuff will be available through the Lottery?

Oh no, we just use different sets of gear based on what we're testing at that moment.

[Kehleyr]

One question: how do you plan to integrate the formal testing with the slightly-washed and usually unruly masses on Isengard? Will it be like +Hengar's to-do list for Bullroarer before the recent update?

Isengard is not a new program. It's been augmenting our QA testing for the better part of 3 years now (the program in fact celebrates its 3rd anniversary this month!).

Isengard is pretty structured - far more so than Bullroarer. Quality Assurance and Devs alike have come to depend on this program for touching on areas of the game and helping us give it a more thorough testing process.

[martin93]

Some other people have brought this up already but pertaining to the "other teams" question i'd love to hear from the world builders. They really are top notch by far. Even a diary about the process of making a simple village would quench my curiosity

[RileyCraven]

Well this diary has brought up a point that I've been struggling with since the Mirkwood release.

Over the past year or so there have been several revamps, ex: bree and lone lands.
It seems to me that point of these revamps was to make the playing experience a lot smoother, another example being that we don't have to continually keep going back to the same area for quests. Aka you get all the quests for an area at once, you do them, come back to turn in and go to the next area, rinse and repeat so on and so forth.

Well it seems to me that these ideals are not upheld in new content. This was especially evident to me in Mirkwood where all of the areas have sets of quests. ie. The Haunted Inn has several sets of quests that keep sending you back to the same area you came from repeatedly. This is really the case for all of Mirkwood and is also very annoying and frustrating for areas that players do not like.

So my question to the devs is this: Is there a process by which you decide that you are going to change older content to be better, you don't have to continually go back and update new content to match the changes you just rolled out for the old? I.e it would be more efficient just to have the new content match how you updated the old content.

If not, why is there not a process in place for this, and if so, how did Mirkwood pass this process when compared to the lonelands revamp?

[WesCrag]

Really loved this 'peek behind the curtain' and I have to add,
you and your QA teams must be the best in the buisness
as LotRO is the tightest and cleanest MMO I've ever experienced.

/salute

[Snogirl]

Thanks for the info! I think it's really interesting to see what goes on behind the scenes. I think you all do a great job!

[Sapience]

World Builders.

I've had that conversation.

[Ayrolen]

World Builders.

The idea of a Dev Diary written by Fantus scares me a little.

[Paulino]

World Builders.

Where is Scenario? I haven't seen him for a while

[Angadan]

Do you have a suggestion for a specific team you'd like to know more about?

Two Groups I'd love to hear from!

* Strategy / Plans - What's the in-game future look like, what's the game plan, spoil it for us, tell us what comes next, and then next, and then next after that as far as the eye can see!

* Finances - How's it working out for you, have enough money for the game to stay healthy for years to come?

[FutenCake]

What good is testing if you don't fix the bugs you find?

Exhibit A: The Forges of Khazad Dum Exploit.

Exhibit B: Mirkwood Beta.

Being QA there must be an incredibly frustrating job.

[arinthros]

Awesome Dev Diary! Great to see what QA does behind the scenes.

In regards to your spreadsheet list, would it be possible to have an opt-in mailing list for Bullroarer that would send us some specific things to look at to try and break?

Alot of the time it's Play through this area and give feedback. If there were a list of detailed specifics to test out, I think it would help some of us out who want to help you all out the best we can.

Just a thought.

[Ayrolen]

Awesome Dev Diary! Great to see what QA does behind the scenes.

In regards to your spreadsheet list, would it be possible to have an opt-in mailing list for Bullroarer that would send us some specific things to look at to try and break?

Alot of the time it's Play through this area and give feedback. If there were a list of detailed specifics to test out, I think it would help some of us out who want to help you all out the best we can.

Just a thought.

http://forums.lotro.com/showthread.p...light=Isengard