The BackpackGearTest.org Bylaws
v. 0609

Appendix 4

The Appendices are provided as detailed explanations and backgrounds of the policies and procedures of BackpackGearTest.org.  Each Appendix contains delineated topic Sections that are not necessarily related.  The Appendices and Sections are provided for reference purposes, and are not required reading. 

The Appendices are written and edited by the Moderator Team of BackpackGearTest.org, with contributions gleaned from the email of the BackpackGearTest group.

CONTENTS

Appendix 1 - How BackpackGearTest.org Works
Appendix 2 - Editing
Appendix 3 - About Rules and How to Change Them
Appendix 4 - Test Calls, Test Applications, Newbie Rule
Appendix 5 - The Apprentice Program
Appendix 6 - All About Report Writing
Appendix 7 - A Lesson in Pictures
Appendix 8 - Copyright, Fair Use, and Links Policies

Back to Bylaws

APPENDIX 4 - Test Calls, Test Applications, Newbie Rule

CONTENTS

1. The Newbie Rule Explained
2. Strategy and the Selection Process
   1. A.  General Discussion - Successful Test Applications - Taken from the Group and reproduced here for reference. - February, 2003
   2. B.  General Discussion - On Applications & Test Plans - Taken from the Group and reproduced here for reference. - August, 2003
3. POLICY - Travel Plans in Applications
4. POLICY - Copying Applications and Reports

1. The Newbie Rule Explained

This section details the Policy known as the Newbie Rule.

A.  Policy Statement - Shane Steinkamp, BGT Moderator

In order to be a tester, you must complete two owner reviews, successfully submit them through the editing process, and post them to the website. You must also fill out and send in the tester agreement. Once you meet those criteria, you can apply to test gear. At this stage of the game, you are considered a 'Newbie' tester until you complete your first test series - three reports over four months. Once you complete your first test series, you are considered to have 'graduated' and are now a 'Veteran'.

During the Newbie stage, you are limited to one test series, with some exceptions. Once you graduate, you can be selected for multiple tests.

The Newbie period is a trial period designed to protect both BackpackGearTest.org and the Newbies. BackpackGearTest.org needs to know that a tester is with us for the long term, and testing multiple items as a new tester can be overwhelming at times. We don't want to loose anyone through short term burnout. Your first test series is also going to set the mark for how much and what kinds of gear you might be selected for, so it is in your interest to do a very good job with that first series since it is the benchmark by which you will be judged for gear selection until you have more reports done. Having too many tests out at once may affect how well you do.

There are some exceptions to this rule, and so you should apply for as much gear as you like even during the Newbie period. Sometimes the Moderators will select a Newbie for more than one test based on performance, demographics, or desperation. Some items, like the Frogg Toggs Chilly Pad I just released, are noted as 'freebies for newbies'. This means that the Newbie rule does not apply to this item, and Newbies can be selected regardless of another test. Freebie items are usually simpler, easier, and cheaper items to test - but your participation in the Freebie tests, which some people see as less desirable, will be noted by the Test Moderators in a positive light. I, personally, also find such items MUCH more fun to test. Testing things like packs, shelters, and sleeping bags - and writing good reports for them - is very often WORK.

In the beginning, the Newbie Rule seems rather hard to get along with. Four months seems like a long time. As some of the other Veterans will tell you, though, their graduation dates did eventually come, and suddenly they found themselves with more gear to test than they could shake a Leki at. (Does the Nuts and Wheelos Gang care to comment? Go ahead, say somthin' Jim.) Testing gear is just like hiking. Only the people that stay on the trail and keep walking get to see the coolest sights and have the coolest experiences. Griping gets you nowhere, while hard work wins every time. Your graduation day will come.

The Editors, Test Moderators, Monitors, Mentors, and a large body of testers are all available to assist you through this process.

I hope that this clarification is helpful. If you have any questions, please feel free to ask them on BGTWeb.

Thanks!

Shane Steinkamp BGT Moderator

B.  Additional Comment - Jim Sabiston, BGT Tester (at the time of writing.  Jim is now a BGT Moderator.)

As Shane implied, I was part of a new group of testers that Shane and company took a chance on a while back. Our first test was the Deuter Futura 32 day pack. (An excellent item, by the way - check out the reviews!). A fairly high profile product sent to a bunch of newbies. As I recall, Shane sweated a bit thru that one, but it turned out all right.

Four months *did* seem like a long time, but once the process began it went very quickly. Once the pack arrived, it was examined more closely than any piece of gear I've ever had which lead to the IR. The end of the first week just rushed up on me.  After that, planning and actually going on the various outings made the next two months fly by and resulted in the Field Report. The next four months were spent rounding out my experience with the pack and experimenting with its limits, great fun. It all passed very quickly.  It didn't seem long until the Long Term Report was posted and I waited for the final edit!

Since then, the list of gear I've tested is making a noticeable addition to the pile in my gear room! It has ALL been fun and interesting.

One additional note: While testing your first item, you can fill some of the time posting additional Owner Reviews. This will give you more writing practice and is great for brownie points (and name recognition) when the good gear comes up and you are a veteran.

You gotta be in it to win it! Get writing!

Jim S.

C.  Additional Comment - Jerry Goller, BGT Chief Moderator

There is no question that the newbie rule draws things out. On purpose. We used to lose more gear than I was willing to accept. Of the various plans we came up with to stop it, the "two Owner Reviews and one complete test series" was the most generally accepted one. It seems to be working. I've very reluctant to fix something that is working. We're not checking for volume, but for length of commitment. We'll look at it, but don't get your hopes up. BackpackGearTest.org isn't going anywhere. It will be here next season with more great gear. All you have to do is also be here....... ;o) Jerry

D.  Questions and Answers - Various.  Answers by Shane Steinkamp, BGT Moderator

1. Are we saying that a newbie, or a 1-2 test participant who doesn't post except for their tests, won't qualify on the basis of being "not committed enough".

No. So long as they meet the minimum requirements, they will be fairly considered. If it comes down to a 'tie' between two testers then commitment may (read 'probably will') come into play. The selection process is EXTREMELY difficult. The more information you give us to make our selections, the better the process will be for everyone.

2. I thought newbies had a snowball's chance in hell, but still a chance, at high priced gear.

They do have a chance - but their reports and applications have to be above average, and they have to swear on the religious test of their choice that they will complete the test. The sad truth is that several people have 'grabbed the gear and ran'. We have to be cautious.

3.  I didn't think the wording so strong (e.g. requirement) to where they shouldn't even bother applying.

They should apply, but they should also understand their chances and adjust their expectations accordingly. With some gear, and especially with expensive gear, we are looking for people with dependable track records. High ticket items, and high demand items are sometimes the rewards for commitment and consistency. Newbies just don't have that yet - but it's not hard to develop.

Remember that we TRY to put at least one new tester into each test, and that often puts new testers ahead of the pack when not too many of them apply. Sometimes, though, that's hard to do for a number of reasons, as noted above.

But, they SHOULD apply! Reason #1 is that we get to see their applications and comment on them before they get too far along in the process. If they wait to post an application on THE item that they MUST have, it's too late to do that. Reason #2 is that on one recent test call I had two newbie testers who were just perfect. I had to pick one over the other. If I run a similar test in the future, the newbie tester I rejected the first time will have an extra 'point' in my book for that test.

None of the newbies should have any problem with any of this. At one time we were ALL newbies. I was a newbie once too, and I'm sure the Moderators said, "No way is Shane going to get that. At least not yet."

Sometimes being a 'long-timer' works against you. Sometimes being a Moderator works against you. I'd liked to have had one of the Grasshopper Child Carriers. I couldn't justify testing one, though, because all the other applications were really good and I felt better about filling the test with other applicants. It's a pretty hard business when you've just rejected YOURSELF!

Maybe next time...

Not all of this is fun and games... ;)

Shane

2.  Test Applications: Strategy and the Selection Process

This section is taken from messages on the BackpackGearTest Yahoo! Group and provides some insight on successful test applications.

A.  General Discussion - Successful Test Applications - Taken from the Group and reproduced here for reference. - February 2003

1. Ron Martino, BGT Moderator at the time of writing.

I've run several test calls recently, and have noticed some variation in both the reviews and the applications submitted by folks. This has been touched on before, but it doesn't hurt to repeat it.

When you apply, be sure to double-check your application. Is everything there? Are there special concerns or interests mentioned in the test call? Have you addressed them? How's your spelling and grammar? (If you show that you are careless over details in your application, I'll wonder how your reports will be.) How's your testing strategy? Is it essentially 'I'll take it into the woods with me', or have you thought about some interesting ways to put the gear through its paces? We aren't expecting the equivalent of a thru-hike, but it's nice to know the stuff will actually get some meaningful use.

You need to include links to at least two reviews or test reports you have already written. Look at them. Compare them to the ones currently being submitted by other folks. How do they compare? If all of your reviews were written under older, simpler rules and you don't have anything current, write an owner review or two. We always need more. It doesn't have to be a major undertaking, but it should be /complete/. These days it's unlikely anything will be posted without appropriate links, date, bio information and such, but how's the review of the item itself? Have you addressed all aspects of the piece of gear, or did you just toss something off in order to meet the minimum requirements?

There's certain folks on the list who consistently turn out good reviews and good applications. They end up getting to test lots of gear. It's also possible that there are some members who don't bother. They in turn may not get to test anything, no matter how often they apply. I hope that won't be the case, but it could happen. It's up to the individual.

I don't mean to talk down to anyone here. I hope this doesn't come across that way. But I do see differences in the applications I receive, and those differences translate into whether or not I end up selecting someone. I want to give everyone a chance to get some of the fun stuff we're being allowed to play with. But I can't do so if their reviews don't live up to our standards. We have a responsibility to the manufacturers to do our best, and those who do will be rewarded for it.

2. Clifford 'Rocky' Haynes, BGT Tester (and Charter Member!)

Ron I under stand that you didn't intent to offend anyone, and I'm not trying to bust your chops. "But"; been with BackpackGearTest.org since before the first Hennessey tester call, and I've heard this before and from other folks. I understand the point you are trying to make, but weather or not folks get gear, or submit good applications is "It's up to the individual." There is a third group of folks that work very hard to write the best application and reviews that they can, but don't often get selected for popular gear or gear that they really would like to test. First there are the standard selection profile- topography, male / female mix, newbe / old-timer mix, and that type stuff. We are growing more moderators and repeat monitors, the preference they rightfully receive effects the weighting of selections. All of these issues affect the selection before the quality of the qualified application becomes an issue.

Not every one has the same education, intelligence (LOL Shane's IQ's higher than mine), or the writing ability. We have feed back in the review writing process the "Edit". A really poorly written report might start with and edit suggesting reading the survival guide, but we seldom totally reject a review. We are about to embark on a mentoring program to improve reports, but we have no feedback from applications other than success in selection.

What I'm trying to get at is weather intentional or unintentional if we lump folks into two groups, folks that do there best and do it well, and people that don't care or try; we could easily be discouraging folks that are trying very hard to write the best reviews they can. If out goal is to be a service to folks that are looking for gear, and to have an influence in producing quality gear we need to be able to review lots of gear. Our excellent editors insure we have quality reviews. So that isn't an issue. But we still need folks that are willing to write owner reviews and reviews on the less desirable gear. Often those people are going to be the folks that are less skilled than the folks reviewing the desirable gear. Perhaps we have grown to the point where we don't need or want folks that don't have the skills but want to try. I doubt that is the case, especially considering we are starting a mentoring program.

Anyway the short of it:: LOL - I would rather be told I'm stupid, than I am lazy. : )

Rocky

2. David Anderson, BGT Moderator

Hi Rocky,

Our goal is to move everyone up to the highest stage possible. You are right, that there are a lot of things that we consider when selecting the testers, but at least with the test calls that I run, it goes something like this.

1. remove applicants that do not meet minimum requirements, really screwed up on their applications, or have lousy reviews.
2. try to narrow it down by demographics.
3. judge by the best applications and reviews within those demographic categories. Balance that with who deserves gear, but hasn't gotten much lately.

You are absolutely correct. Those demographics can be a killer, especially for those of us that are fat, male and live in an area where there are other testers.

For the sake of the newbies here, I will say that we try to get one newbie in to each test. But the definition of newbie does depend on the test. First time testers are not likely to get a $400 tent or sleeping bag. But if they do a good job on their first test, they might get the newbie slot on that gear, or it might take a couple more tests for that to happen.

We also like to make sure that we get women testers in every appropriate test. For a long time this meant that the women got a lot higher percentage of the gear that they applied for. But now, it looks like we have a lot more women coming in. They may finally have some competition :-)

Overseas testers also face a few extra hurdles. Some manufacturers don't want to ship overseas. We try to change their minds and give them some options that might work for them. But even if they agree, we generally need to limit it to one tester unless the item is already in full production and in the hands of their overseas distributors already. We also hope that with the reversed seasons we can interest US manufacturers in testing in the southern hemisphere to have their reports out in time for "the season" up here. We are also open to the possibility of running tests of Australian or European gear where the overseas tester would actually be in the US. We Will have to work our way into all these options. So far, we have been able to get enough of the gear overseas to keep the testers fairly happy.

As for the discrimination against us big fat boys, all we can do is keep reminding the manufacturers that we are out here. There is a lot more gear coming out for us big guys, and a lot of it is from manufacturers that we have lined up. We probably aren't going to get to start with their big guy gear, but it will eventually happen. They want their big sellers reviewed first.

Finally, we will be running another one of those test calls like we did last spring, where we offered a critique of the applications as they come in. This will hopefully help the newbies see what is expected of them. This will also be a part of the mentoring program once Shane gets it going.

As for the gear that goes to the moderators, repeat monitors, top testers, and various other old-timers. There is going to be a lot of new old-timer slots available. We have even more of the popular items than ever before. Tents, hammocks, sleeping bags, packs, trekking poles and lights. Everyone will get a chance to work their way up. Even with the demographic limitations, everyone who is willing to put in the effort to do a good job will get to test gear.

Hopefully this all makes sense, as I'm writing it as 3 a.m. is closing in on me.

Dave

3. Andy Mytys - BGT Tester

First... no gear should be viewed as "less desirable". These items you refer to should be viewed as a "challenge". Go take a look at my recent flask, mitten, and bucket reviews (upload still pending, but posted yesterday) for an example of "less desirable" gear with fleshed-out, thought provoking, reviews. It can be done.

(Note, Andy is here referring to Rocky's post here, where Rocky makes the statements below.  Rocky's text is in dark blue.)

For those that want the point quickly, it's "When I feel that I'm no longer participating, I just write an owner review."

For those that just get posts via mail, here's the text in question:

I consider myself a successful applicant. My Strategy is simple.

1. I don't write well, can't spell, and type with one finger.

2. I don't fit the manufacturers hiker profile. Most manufacturers aren't interested in selling gear to fat people.

3. I'm not a real likeable kinda guy. On more than one occasion I have tried to make a constructive comment, and had the comment seen as criticism, and dismissed as such.

OK I should have made my point, I'm certainly not the most chosen applicant.

But since my interest is to participate in the list, it's up to me to find a way. When I feel that I'm no longer participating, I just write an owner review. I don't have to compete with the better report / application writers, I just have to write the best report I am capable of and get it good enough to be accepted. I didn't lower the bar so much on the quality of the report, I just avoided the competition, for the opportunity to write the report.

The other key to my success as a test applicant, is that I have applied for less popular test items ( provided I felt that I could give it a good test).

My chances are a lot better to be chosen if there are only a couple of applicants for each piece of gear. Maybe I don't measure up when there are 6 or 8 applicants for each piece of gear, but by going for the less popular I get to participate.

I have been fortunate to get some really good stuff, that lots of folks missed out on. For example if I hadn't applied for the SuperFeet Quick Fit test, I'd be sitting home tomorrow nursing a leg injury instead of hiking.

Sure, I wont summit and I'll be on crutches, but the QuickFits have reduced the pain to the point that I can go. The Doctor has taken pictures and given his usual it's going to hurt till it stops but you can't hurt it by using it.

Another piece of gear has been great fun to test, I never would have considered it for backpacking, but I use it quite often. This particular piece of gear opened an opportunity for me to Beta Test a piece of gear, that as far as I am concerned is the best piece of backpacking gear I have come across since my Hennessy Hammock (which I purchased so I could participate in testing as an owner). Had this piece of Beta gear been in production and been tested on the list, it would have been popular enough that I would not have had a chance to test it. Frankly even though the reviews would have been good, I don't know that I would have bought one to try. It didn't fit my backpacking style. Well it's changed my style, and made my pack lighter. Now I'm hoping it goes into production soon so I can buy one for every one in my family.

The bottom line is, a successful applicant, isn't necessarily the one that collects the most tents and backpacks. The true measure of success as an applicant / tester, is the value placed on your report by the reader ( the end user). If you have honestly tried to do your best on a report, weather be an owner review or a test item, and you someday receive a note of appreciation from a non-list reader, that note of appreciation will say more, to you, for your success, than being chosen as one of the 4 people to receive a backpack sixty people applied for.

4. Shane Steinkamp - BGT Moderator

> There is a third group of folks that work very hard to write the best application and reviews that they can,
> but don't often get selected for popular gear or gear that they really would like to test.

True. There are also people who will NEVER actually get to participate in a test. I think that number is pretty low, however...

> First there are the standard selection profile- topography, male / female mix, newbe / old-timer mix, and that type stuff.
> We are growing more moderators and repeat monitors, the preference they rightfully receive effects the weighting of
> selections. All of these issues affect the selection before the quality of the qualified application becomes an issue.

All true. However, I slightly disagree with the last sentence. Someone could be a great monitor, and still submit poor reviews. They won't get gear...

> Not every one has the same education, intelligence (LOL Shane's IQ's higha than mine), or the writing ability.

Perhaps, but IQ has little to do with it sometimes... I've applied for gear that I haven't gotten...

> We have feed back in the review writing process the "Edit". A really poorly written report might start with
> and edit suggesting reading the survival guide, but we seldom totally reject a review. We are about to embark
> on a mentoring program to improve reports, but we have no feedback from applications other than success in selection.

I think that if I saw a really poorly written application from someone who was trying hard, I would try to work one-on-one with them to improve it. They might still not get selected, but they'll get help.

> What I'm trying to get at is - whether intentional or unintentional - if we lump folks into two groups,
> folks that do there best and do it well, and people that don't care or try; we could easily be discouraging
> folks that are trying very hard to write the best reviews they can. ... Perhaps we have grown to the point
> where we don't need or want folks that don't have the skills but want to try. I doubt that is the case,
> especially considering we are starting a mentoring program.

I want to address this, because I think it's a very valid point. To sum it up, there are people who get rejected - and rejected a lot. There is no recourse for these people now, except to throw themselves on the mercy of the Test Moderators and try to get some help.

The Apprentice Program does address this. People who get frustrated can say, "I've applied ten times and never gotten gear! What's going on!?" They can ask for a Mentor. That person will work with them OFF LIST, ONE-ON-ONE and be able to tell them, as the case may be, "Well, both of your owner reviews suck. Let's work to make them better, and lets do one or two more to make sure you have the knack." After that, they can work together on the application, ironing out any rough spots. Even after they are selected for a test, their Mentor is still with them, checking their reports and making suggestions. Help is never more than an email away.

The PROBLEM with it is this: Once all of the reports are perfect, all of the applications are perfect, and all of everything is perfect, then what? All the same things that you said apply...

In the end, a Test Moderator is faced with a very real challenge - and sometimes an ethical dilemma. I had a problem running the RailRiders Test Call. Andy and Dennis had the same demographics. They write similar reports. They had similar applications. They have both received similar amounts of gear from the list recently. I like both of them. So, what to do? I could have looked at both applications and counted mistakes like commas out of place, but I didn't. Instead, I flipped a coin. Andy lost. Was it a big loss? No, because Andy is very involved. I won't hurt his feelings, and he'll keep on keeping on. He'll keep applying for all kinds of things and he'll get gear for a number of reasons.

The BIGGEST reason, however is that he IS active, and he DOES apply.

That's the biggest secret of all: YOU CAN'T GET GEAR IF YOU DON'T APPLY! I've been here awhile (not as long as Rocky), and I see the same people apply over and over again. I've also seen new people come in, fill out their owner reviews, qualify, make ONE application, and then BITCH and LEAVE when they didn't get selected. That's just not the way the game is played...

In the end, BackpackGearTest.org is not about the gear. If I never get another piece of gear from the list so long as I live, I will still be here participating if my life circumstances allow. Why? Because BackpackGearTest.org is important to me for a number of reasons. One, I like you people. Two, I think that what we do here is important in a very real way. It improves the chances that people will select gear that is right for them. It also lets people make educated decisions about gear that they may one day depend on for their very LIFE. I really don't want to find out that a sleeping bag isn't worth it's rating by freezing to death in a blizzard... If BackpackGearTest.org was just about gear, I'd leave...

> Anyway the short of it:: LOL - I would rather be told I'm stupid, than I am lazy. : )

Nope, neither. In any case, the mentoring program will be able to fix IGNORANCE. LAZY we can work with to a degree. STUPID won't last long anyway. They'll go get themselves killed off a cliff or eaten by marmots...

Your points are well made and well taken Rocky.

Thanks,

Shane

B.  General Discussion - On Applications & Test Plans - Taken from the Group and reproduced here for reference. - August, 2003

1. Shane Steinkamp - BGT Moderator

> What we don't want to turn this into is an exercise in "one-upmanship" where you read all the applications,
> glean everyone else's ideas, then tack on a few new ones and think that yours is better.

The Mods would definitely catch onto something like that.

> The challenge is to have your app in first, and be brilliant.

I want to make a clarification for the general benefit of the membership.

Quality is often more important than quantity. Make no mistake: Testers are in competition with one another in the application process.

That doesn't necessarily mean, however, that your application is in competition with another application - or that there needs to be any kind of insane one-upmanship.

It is enough for your test application to stand solidly on its own, and in like manner, your test plan should stand solidly on its own. It isn't necessary to try to pre-write the initial report, or to list in great and minute detail every aspect of your testing. "I will count the teeth per inch of the zipper to make sure that it is within specification.", is certainly going way too far.

You shouldn't concern yourself overly with other people's applications and test plans. In the end, they may have nothing to do with your circumstances. I get lots of rain in my area, and you may get none - so your test plan won't include rain. Mine won't include desert conditions.

What *I* look for in a test plan is an awareness of what the product is, some possible limitations to test, and a plan that will let me know what kind of field experience a tester will get with the item.

Using a tent on day hikes and setting it up in your back yard for a couple of nights is NOT an adequate plan, for example.

When I'm selecting testers, I always also look at reports. I will always want to have at least one tester who is going to go into the minutia of the item - like my reports do - but I am also going to pick people who do NOT write those kinds of reports - people who focus more on how it 'feels' and how it 'flies'.

In the end, everybody should be the best tester he or she is - and stick with that. I don't expect some people to break out a caliper and check the OD, ID, and wall thickness of tent poles, but there are others that I do.

I guess what I'm trying to say is that nobody should be trying to out-do anyone else as far as test applications or reports go. Yes, you should try to reach for a high level of quality and thoroughness; but each person is going to have a different level of achievement in that regard. Be the tester YOU are, don't try to out do anybody - because when it comes to the tester selection process, you might be out doing yourself. ;)

Not to pick on anybody, but Fuzzy's application for the Snugpak is exactly what I'm looking for. It covers the bases, is solid in its plan, and lets me know exactly the kind of use I can expect the bag to get. Without looking at all the minutia of grammar, spelling, and conversions, it's a good solid report. Just what I expect from Fuzzy. That doesn't mean that he will automatically get selected, though.

Remember; we aren't selecting applications, we're selecting testers. While the application is very important to the process, all of your previous reports - and sometimes previous applications - come into play as well.

Shane
BGT Moderator

2. Comment from Tester, and follow up by Shane Steinkamp - BGT Moderator

> You'll find lots of different styles and viewpoints represented, and lots of folks
> here willing to give feedback, to help you develop your own unique testing style and perspective.

And that is, in the end, the real goal. You've really nailed it Steve. If everyone generates the same reports, then it's just as well to test one item. We want the differences. I think it's really interesting, for instance, looking at the recent Child Carrier tests. I love mine, and won't ever use a stroller again. Someone else, on the other hand, had the opposite reaction. Our value as a group lies in large part in the verbalization of the differences as useful information that others can use to make decisions.

Shane

3. Additional Commentary - Jim Hatch - BGT Moderator

Maybe this deserves some more detailed treatment in the survival guide. (Note: Here it is!  Ta-da!)  I think there are competing issues at work here. First, for newbies, without a series (or multiple series) of tests & reports behind them, the application is going to be their major foot forward as it were in terms of their writing. And they're going to be looking at it as a competition to get the test(s) that give them the opportunity to get out of newbie land.

I don't believe the ORs give as much opportunity since they're an after-the-fact review of the gear. I know when I write an OR I try to identify what I was looking for in the general gear type (stove, light, knife, etc.) and why I chose the specific piece of gear I bought. Then I report on whether it met those criteria or not & whether I'm still using it and would buy it again. The test plan comes after the choice not before.

For BackpackGearTest.org gear I approach it from a more methodical engineering perspective (I'm not sure if engineering makes you anal retentive or if you need to be anal retentive to study engineering).

First, I print out the product info from the company's website and highlight all of the product features & performance claims. I then create test cases
(scenarios) to validate or refute the company's claims.

Then I take a look at environmental factors - winter gear should be tested with gloves because lots of things work ok with bare fingers but not gloves, in extreme temperatures cause stuff doesn't always work at the extremes of temp (either hot or cold), in wet...etc. etc. Summer gear is similar in that there is the effect of sun, high temps, high or low humidity, increased water needs, etc.

With the environmental tests identified, I look for personal-specific things. For instance, as a hammocker, there are unique demands placed on pads, bags, etc. As a lightweight, low-volume backpacker I look at the impact the gear has on a 1lb pack, what's it going to do to my total weight/pack volume, what features would make it worth carrying the extra weight/volume (I'm starting to focus more on volume lately now that I've got weight down to a reasonable level -- at least that's what I told my wife to justify the god-awful price I paid for an Arc Alpinist rather than be happy with the warm & toasty but bulky bag I already had). What I'm looking for with these features/tests is how the gear works in my gear system.

Then I check out any other applications that have already been submitted. There may be things in other ones that I think should be part of a comprehensive test. There will be things that I've already got covered (after all there are just so many things to do with a bag or a pack). There will be things other people think are important or matter due to their system that doesn't matter to me (like people who camp in ice caves or the desert or try to use towels as loin cloths - sometimes other people's ideas just seem silly to me 'cause no one looking at BackpackGearTest.org is really going to do the same...extreme creativity may be called for on a Wechsler test but isn't always a good thing in the real world).

Then I usually look for a humorous perspective on the gear -- we're not trying to avert war or something here, it is only gear testing after all. I also look for some feature that would "require" me to buy a new toy so I can rationalize the purchase (my wife claims I only do projects around the house that I need a new tool for...same thing here). It's just part of what I get out of the test. I don't *need* a Kestrel, but now I *want* one and I'm going to find a reason I *have* to buy one.

When I'm done, I have a detailed understanding of how I'm going to approach a test, have a good sense that what I've got planned will achieve the objective (validating or refuting the company's claims and my preconceptions), and a plan I can execute from the day the stuff hits the mailbox. At the end of the test I can go back (as I'm now doing with the Moonlite) to check to see if I covered everything I intended or if I missed anything. It helps keep me from forgetting something "in the heat of testing". The test plan becomes my personal checklist.

Maybe that's more detail than the Mods want, but I think it's a valid approach. If you guys don't want to see all that, I would still approach it the same way (a "private" app if you will) and do a consolidated version for the official app. Some additional guidance in the official doc would help point me & other newbies in the right direction vis-a-vis test mod's expectations.

Jim

4. Additional Commentary - Andy Mytys - BGT Tester

[In my opinion]...[N]ewbies would be far ahead by having a basic application and six solid owner reviews posted, than just two ORs and a bitch'n app.

> I know when I write an OR, I try to identify what I was looking for in the general
> gear type (stove, light, knife, etc.) and why I chose the specific piece of gear I bought.

I don't do this at all, as I'm a rather impulsive shopper. Fortunately for me, I do know good gear when I see it, and I fiddle with whatever I buy at home before taking it out into the field. Needless to say, there are a lot of returns in my house. Returns are usually because of fit/comfort, and have nothing to do with quality. Why do I buy gear? Usually, because I needed it, it looked cool, or it shaved a few ounces. I do have a bag fetish - bags I buy just out of the coolness factor, which is always about compartment design and placement.

> Then I report on whether it met those criteria or not & whether I'm still using it and would buy it again.

I don't comment if I would buy again or not... I just write my experience, and let the reader decide.

In my applications, I don't think of test cases for the individual product as much as imagine where I might be in the next four months, how I would use the product, what challenges I might run into (e.g. wearing gloves), then make the list. I also look at the website, but only to see if there are any unique product features. I don't think "how can I test that feature," in terms of wanting to come up with all the possible test scenarios (like an engineer would). Rather, I ask myself how the features would integrate into my style of hiking, then I list the questions I have, as to how I will use the feature and what question I want to answer as a result, in my app.

There's no reason to be an engineer, and list ALL COMBINATIONS of possible uses. That's why we have multiple testers. Just integrate it into your usual routine, and report on it. Sometimes, your routine will change due to the product's performance. You'll comment on that as well.

I made the "loin cloth" comment in my app. Yes, it was silly. If you looked at the manufacturer's website for that towel, they listed a lot of outlandish uses. I think that, when a manufacturer goes out of their way to claim how a product is used, we should try to test as many of those claims as possible... not only test the product in our own field uses. That's the one exception for me. Now, as I go off trail, in places where there's nobody for MILES, I've got no modesty issues to consider.

I think detail in a test plan is a good thing. It can't hurt your app, that's for sure. In my case, I'm have a pretty high success rate in my applications. I don't apply for too much "popular" gear, as I don't NEED it personally. I like to test gear that's multi-use, and that's usually the small stuff nobody else wants. I look at myself as a BackpackGearTest.org resource, and if an existing test has lots of good applications I'd be wasting myself as a resource if I got selected, IMO.

<SNIP>

I have nothing against thesis applications. I printed out my HighGear Axis Watch app - 7 pages long. I'm just saying that they are not required. They are simply a reflect of someone's style. With that in mind, if any applicant is trying to outdo the application of another, by simply making their app longer or more detailed, it's not worth it. Better to put in a simple app that you're comfortable with, and spend the extra time it would take to "fatten up" your app writing and posting another owner review or two.

For some of us, myself included, a short and concise app might actually take more time... because it's not the way we think. I'm one that likes to do "braindumps", then take all the ideas and put them into a logic order that reads well. There's your app. If I had to stay within 1000 words, that would indeed be work! :)

Andy

4. Final Word - Jerry Goller - Chief Moderator

I'd like to clear up something myself. The applications aren't, or shouldn't be, the newbies major foot forward. Their ORs should be. Their ORs are where they should be demonstrating their writing skills and their reporting skills. This is where they should be showing their powers of observation and evaluation and their ability to convey those observations to the reader. This is where they should be putting their major effort and where the Test Moderators should be going to gather the information required to determine the testers abilities. The application should only be used to provide the biographical information necessary to aid in the tester selection. The test plan is only to demonstrate that the applicant understands what the item is and how the manufacturer intends it to be used. The test plan also should show that the tester will have the time and be in the applicable element to properly test the item. The application shouldn't be the forum for demonstrating the tester's abilities. That is what the ORs are for. The applications should be just that...an application with sufficient information to select the tester for the physical requirements of the test and the links to the tester's reports to provide sufficient information to select the tester for their writing and reporting ability.

Jerry

3. Travel Plans in Applications

This section explains the policy relating to Travel Plans in Applications.

A. Policy Statement - David Anderson - BGT Moderator

We have noticed in some of the recent applications that some of you are planning trips of more than a couple of weeks this summer or fall. When you are applying for a test, please make the dates of these trips very clear, and take note of where these trips fall in the probable schedule.

While we never know exactly when the gear is going to ship, most of the time we can come up with a reasonable guess. While being gone around the time that a report is likely to be due will reduce your chances, other factors might still cause you to be considered. If you are going to be gone for a month, but don't tell us which month, then you are almost certainly going to be eliminated. If you expect to have regular Internet access on your trip, you can let us know this too.

For example if you are going to be gone from the middle of June to the middle of July, then April is not the time to be applying for tests. If shipping gets delayed and causes conflicts, we can always work it out at that point.

A1. Policy Clarification - Shane Steinkamp - BGT Moderator

If testers are not comfortable posting away dates in a public forum, they may send this information OFF LIST, directly to the Test Moderator or Test Manager who ran the test call. 

B. Response to Questions - David Anderson - BGT Moderator

1. Can the moderators meet the testers half-way and publish the "reasonable ship guess" in with the test call?

Answer: You should be able to figure this out by yourself by now. 1-2 weeks to complete the test call process. 0-3 weeks for them to ship, up to 1 week to arrive. 2-6 weeks from the test call is the most common time frame.

2. Rather than worrying moderators with when my trips will be, I could bypass applying altogether, based on the target ship information provided. If product deliver is scheduled for August, a month that I may be out for, and the test call goes out in June, I won't even waste my time with an app. In today's scenario, I'd apply because I would assume that the ship date would be sometime in July.

Answer: It has little to do if you are there to receive the gear, though that is important. The concern is that you are able to file your reports on time. Yes, I want the applicants to think about their schedule themselves, but you should still tell us this information so that we know what to expect when the ship date slips.

3. I've noticed that the time frame between test call and test until delivery can be, at times, MONTHS. If the moderators know this in advance, why not post it?

Answer: We don't. I don't know how you came to conclude that a "reasonable guess" means that we know that the shipping date might end up 4 months out. If we know it was 4 months out the test call would be 3 1/2 months later.

4. Also, testers are required to take into consideration work load and availability of time BEFORE applying to test a product. In my current case, I have two test products I've been selected for that have yet to arrive. I am not applying for any more tests, as I don't want to risk falling into a situation where I have three or more tests to deliver in a given week. I am simply not comfortable with it. If a moderator told me, on the other hand, that I wouldn't see any test units until July and another test was announced with a ship target date of late May, I would be comfortable in applying for the test.

Answer: There are no test calls where we expect the ship date to be months out. But it happens.

4. POLICY - Copying Applications and Reports - Jim Sabiston, BGT Moderator - January 2005

This section explains the policy in regards to copying another's work.