Amazon Live App Testing Pain

Amazon Live App Testing Pain

Finding and installing our Android test apps on the Amazon Underground app & the Amazon.com mobile site
was unnecessarily difficult. My workaround is to open test invite emails on the desktop and then add apps to wishlists to be able to find them on mobile devices more easily.

Amazon has an app testing sandbox environment called “Live App Testing” that is similar to Apple’s Testflight and Google Play’s Alpha/Beta testing. Each come with their own set of issues,  but for me, Amazon’s  LAT is the most  problematic.  Sure, Apple must approve each and every build for external beta testers and Google Play makes testers sign up to Google Plus and join a community. But Amazon makes the process particularly difficult by not listing your test apps directly in the Amazon app or mobile site on testers’ devices.

I won’t bore you with the minutiae of setting up your test- that part is straightforward enough, and there are lots of tutorials out there. But once you get set up and start a test, it can be  difficult to find and download your test apps from Amazon unless you know where to look. The test apps do not seem to show up on their mobile site nor in the Amazon store app or even in its replacement, the Amazon Underground app,

Let me digress for one second about Amazon Underground. Apparently now, all new users are required to install AU in order to download  any apps from Amazon whether they are Underground apps or not.  (Amazon Underground apps are completely free – even the IAP.  Amazon pays developers for the amount of time users spend using the app, and I believe they serve only Amazon ads.),

Getting back to LAT, when you start a test, an email is sent to your testers with links to your app in the various Amazon stores worldwide. If users click on those links from their mobile devices, they get taken to the homepage in the Amazon mobile site where the test app is, seemingly, nowhere to be found. If it is there, it is buried pretty deep.  Open the links  on a desktop browser on the other hand, and you will see the app right away, together with  A prominent, yellow, “Continue” button and a grey “Add to List” button below it, similar to other items in the Amazon store.

The natural inclination is to click the “Continue” button. But doing so just takes you to a page advertising Amazon Underground.  There is no indication that you have ordered or downloaded the app or what to do next.  The next logical inclination is to go back to your mobile device and install the Underground app and look for the app there. But you still won’t find it- at least not without some digging.

Back on the desktop site, if you look under Your Account / Your Android Apps and Devices, you will now see the app, but still no indication for downloading it onto your mobile device. There is, however, a big grey “Add to wishlist” button. By clicking that  I was finally able to  easily find my wishlist in the Amazon Underground app and from there find and install the app. It may have been possible to just click the “Add to List” button in the first place and skip the “Continue” rabbit hole altogether.

Okay, solved. Just use the “Add to list button.”

But wait, there’s more! Somehow, despite uploading a new binary to my test, the downloaded version I tested was still an older one.  In order to force an upgrade, I had to delete the old binary and start a new test before the new one would work. And doing that is tricky as well- you have to click a button to upload a new binary before you see buttons to delete the old ones.  And to make matters worse, you can upload more than one binary per test- presumably to test different binaries on different devices. But there is no indication of which devices get which binaries. What a mess!

My advice is to stick with Google Play for Android sandbox testing unless you are using Amazon’s IAP and GameCircle.