You need to know what's on the cutting-edge of technology. Find out what's coming and the unique Warptest POV with just one click on the "Blog" tile.

Blog

Mobile Testing Is Different

Mobile Testing or Mobile QA has taught us that the traditional testing approaches and methodologies need tweaking or changing. What worked as a defined process for testing desktop, client-server or web applications didn’t exactly fit mobile.

If you are a tester who is just starting to test on mobile or simply want a fresh perspective (even dare I say it a mobile Developer) then this is for you. Of course, if you are simply interested in the subject feel free to read on as well.

12 Critical Mobile Testing Issues:

  1. Fragmentation: as a tester you may have to support not only Android and IOS but, multiple devices and OS versions. Android especially is a market with hundreds of devices of varying specifications. Screen size, resolution, processors, memory and more all have impact on device performance and behavior. Now factor in that Apple and Google both release new devices at least once a year but, depending on the countries in your market, there may be many Android OEM devices you want or need to support.
  2. Mobile Device Labs: if fragmentation is a problem, this is a solution worth investigating. Some testers have their own devices onsite but for many who don’t have the budget, or want a more expansive solution then SAAS solutions where the tester can connect to live devices thru a web application this delivers nicely. The tester can choose the devices and with some solutions can run both manual or automated test scenarios.
  3. Compliance: mobile testing of apps requires the tester to be fully familiar with the rules of the App Store and Play Store. (Of course there are other stores beyond just Apple and Google) In addition, compliance has grown to include GDPR and Disability support.
  4. App Types: Mobile allows us to create apps of different types and technological base: native apps, mobile web apps, hybrid apps and more recently Progressive Web Apps (PWA). Each of these has their own testing challenges, scenarios and scope.
  5. In-app Purchase: IAP is a niche in E-Commerce where the customer can make purchases from within the app. Examples can be E-books, multimedia, game purchases (allowing the game player to progress in the game faster) and real-world items. IAP has it’s own compliance standards for each store and can be more complex to test that run-of-the-mill E-Commerce.
  6. Alpha / Beta testing: if you are testing an app intended for the App or Play Store then how do you get this into the hands of a small group of alpha or beta testers without publishing the app? Apple allows Developer Account holders to use Testflight to do this and Google has Play Store Alpha / Beta testing. Once these apps are published and distributed to your selected testing group, you can easily publish to the Stores on completion of your test review or roll to a new bug-fix version as needed. In the case of testing features like in-app purchase, the tester needs this option to facilitate test purchases.
  7. Crash Logs and Analytics: mobile testing requires a skilled tester to have a strong familiarity with logs on devices and services that provide crash and user analytics. Crash logs can be accessed from test devices by connecting IOS devices to XCode and Android devices to Android Studio. There are other methods but these often are the fastest when you want to add the critical information to the bug you are reporting. Crash analytics are frequently a SAAS solution that will show the tester frequency of bugs, and which devices or OS version(s) the bug reproduces on. Each provide actionable intelligence at different scales. Crashlytics is one popular example which can be added to your apps.
  8. App Distribution: there are cases where you may not want to publish an app to the App / Play Store or you want to distribute an app for internal or enterprise use. In these cases solutions like HockeyApp, now renamed and revamped as Visual Studio App Center (after being acquired by Microsoft) offer a way to do so. App Center combines several of the other solutions listed here and is cross-device and cross-platform.
  9. Automated Testing: testers will ultimately seek out a framework that enables them to test UI and function, both rapidly and repeatedly. Appium, Protractor and others allow these testing scripts to be run on  real mobile devices. Automated testing on mobile has its own challenges and ROI. There are also cases where this will not work.
  10. UI, UX and User tolerance: these 3 are interconnected facets of the same issue. Ultimately users have a lower tolerance for poor UI or UX and are willing to remove an app that displeases them. Android and IOS have best practises for design and UI frameworks of their own. E.g. Material Design on Android OS. Testers should be aware of the design standards, what are acceptance criteria and be able to identify UI / UX bugs that can compromise the quality of the mobile app.
  11. Reverse compatibility: As mobile OS versions advance with time certain features, SDKs or APIs become deprecated and or out-of-date. Features and functionality that worked in previous versions may then stop working due to these changes. One example was the way IOS handled webview in Native apps; after IOS9 this changed (due to a major overhaul to remove serious bugs). Some developers and companies reduce this overhead by supporting only those OS versions after such a change, others only support the latest version of the OS.
  12. Non-functional testing: mobile apps live and die on their scalability, app performance and transparency if they function on low-spec mobile devices. This type of testing should play a critical part in optimal mobile testing to prevent nasty surprises when your app goes to production.

Mobile testing - infographic

The Warptest POV

Mobile testing shares as much as it doesn’t to other platforms we test on. The biggest issue is perhaps how rapidly the mobile landscape changes. As processors, GPUs shrink and battery life grows, screens lose their bezel, handsets lose earphone jacks or data / charging cables change their connectors. Meanwhile, wireless charging is becoming mainstream and of course camera resolution increase. Augmented, Virtual and Mixed Reality impact the capabilities, functionalities and spec of mobile devices.

Mobile testing has to evolve as rapidly as the devices we test on.

This post is a sampler or taste-test. The solutions listed are not exclusive, each exist in a competitive ecosystem which mobile testers need to be continually learning about. Some solutions are offered by Apple and Google, others by 3rd parties or Microsoft. Hopefully this is a starting point that wets your appetite for exploration. I’m always happy to hear from you about alternatives and learn something new.

Some of what I have written about will get their own, more detailed posts or Vlogs so keep watching this space.

The Microsoft GitHub Acquisition Is Official …

The Microsoft GitHub purchase is happening. The story leaked before the weekend onto the interwebs and by Monday June 5th, Microsoft and GitHub came clean with their official announcements.

The real loser in all this strangely enough was Apple whose annual WWDC event was fighting for attention online against the pro and anti-acquisition groups. GitHub and their competitor GitLab are more than just collaborative source control. Both have been adding features over the last few years including issue tracking, stories and other features evolving these tools into ALM solutions.

Why would anyone be against Microsoft acquiring GitHub?

After asking this question online, the only coherent, meaningful or logical argument I got was nothing to do with Microsoft and everything to do with any tech giant acquiring one of the most popular collaborative source-code tools on the market. To be fair, no-one should be surprised by this:

Microsoft GitHub - Reasons to be happy

Microsoft had been heavily invested in Git for several years, making it a fundamental option for working in Visual Studio. Meanwhile, everything about the Microsoft GitHub purchase is about mutual benefit but more importantly, increasing value for the Developer Community.

The two other reasons I got out of developers, startup founders, Devops and other were: –

  1. It’s Microsoft. I don’t trust them.
  2. Look at the acquisitions Microsoft ruined.

The counter argument to these were simple; what’s not to trust. Since Satya Nadella took over Microsoft, the company has radically changed. Look at the acquisitions since Satya Nadella took over as CEO. These companies have flourished and grown as part of Microsoft and so have their products e.g. LinkedIn.

If you really want to deep dive the reasons why Microsoft is a totally different company and its context vis a vis GitHub then you can check my video on it:

On the one hand, those against the acquisition are really stuck in the past. On the other, it’s a free market and you can vote with your feet. Tamir Gefen, CEO of ALMToolbox pointed out that this has happened with a major migration to competitor GitLab, who haven’t been shy about pushing how and why you can make the jump. He points to live online graphs of a large upturn in migration onto GitLab from their Grafana:

The graphs show migrations but don’t show some important metrics like: size of repositories, number of accounts moved and from what price bracket these migrations occurred. If the bulk of these are individual developers or people who opted for free accounts then the impact to GitHub will be marginal, other than the bandwidth and API calls being used in these migrations.

The Warptest POV

The Microsoft GitHub acquisition serves to highlight the strategy of Satya Nadella’s Microsoft. This is a break away from the Ballmer era. Microsoft is the biggest Open Source contributor in the world today (including projects like GIT VFS). Microsoft contributes to many projects like Electron and you only really need look at the Microsoft GitHub account to understand the scale of investment in developer hours that led to this purchase being a logical step.

Microsoft’s own online user documentation is held on GitHub repositories. Meanwhile Redmond continues to build more cross-platform apps for mobile than anyone else. They can do this because Visual Studio supports development in most, if not all platforms. Which lead us to other successful acquisitions that were logical steps to bring us this place;the purchase of Xamarin and HockeyApp.

Xamarin has flourished with Microsoft and their former CEO, Nat Friedman will takeover running GitHub following the purchase. HockeyApp has been rebranded and rebuilt by Microsoft as Visual Studio App Center.

This slideshow requires JavaScript.

Microsoft is fully aware that they need to earn our trust, even if the days of Vista are over. BUILD 2017 and 2018 showed us that Satya Nadella has placed the Developer Community, in its entirety front and center. Microsoft is a company that listens and continues to innovate, by purchasing companies and technologies that add value for their employees and end users. GitHub was also in talks with Google but at the end of the day opted to move forward with Microsoft.

For what it’s worth. if you don’t have to jump ship from GitHub, then my advice is to be patient and you will see why Microsoft and Satya Nadella have already earned our trust.

One final thought, if Microsoft now has Xamarin, HockeyApp and GitHub. What impact is this going to have on Visual Studio TFS in the future?

Microsoft GitHub - Keep Calm

Whatever happens. Keep calm and carry on Git Pushing.

GDPR IS LIVE

GDPR like any legislation has unforeseen consequences and impacts. Not for nothing have we learned how little Congress and the EU Parliament comprehend of Facebook or the underlying technologies of the Web, based on their questions when confronting Mark Zuckerberg.

GDPR - Zuckerberg vs EU

These same people in the EU have legislated GDPR. Many will not know exactly what GDPR (The General Data Protection Regulation) is. WordPress provided a guide and template for compliant Privacy Policy that you can see here.

Unless you are currently residing in a cave, your email inbox is probably swamped with mails from websites & apps you have signed up with in the past asking you to opt-in to their updated, GDPR compliant new ToS.

Last night I shared this video on Facebook Live with my take on GDPR and what might be some issues that did not get proper consideration.

 

GDPR & Software Development

Over the last 10 to 15 years, the development culture and lifecycle has been strongly influenced by Web 2.0 both for web / mobile apps and sites. Methodologies also evolved based on technology and platforms to ensure rapid product delivery. Agile & Lean replaced Waterfall methodologies and the culture included these interconnected practices: –

GDPR vs Development Practices

Ultimately MVP (Minimum Viable Product) and the ethos of let the customer test it were based on rapid delivery or a just ship it mentality. Some of this was based on fear of not being first to market, some on the fallacy that software testing was a bottleneck.

Beta testing also allowed release of an incomplete version of the software to a sample group of end users to evaluate customer satisfaction with the product and its features.

These fly in the face of GDPR compliance. De facto, data protection is something that should be baked in from the start and until this becomes habit, most will develop their apps and sites and tack-on data protection after the fact.

Where does MVP go from here?

The Warptest POV

GDPR and the just ship it mentality of MVP are in conflict. Apps and sites will comply because the alternatives are limited. There are companies already opting to block European users until they can be sure they are compliant. Just today, the Verge reported that Instapaper has done this temporarily. We can be certain that smaller developers and startups may opt to keep their apps out of the different European countries App Stores, Play Stores or Windows Stores as the scale of fines are greater than the accrued benefit of onboarding European users prior to compliance.

This leads to some serious questions. What are the geographical boundaries of this law?

  1. What if an EU citizen uses a VPN or some other method to bypass geo-blocking to download an app? Is the app still liable for any violation?
  2. What if an EU citizen is on vacation outside the EU and downloads an app or surfs to a site? Where does the EU see their jurisdiction ending regarding data protection of their citizens?
  3. Microsoft announced that they want to make GDPR the standard for their worldwide operations. Will we see GDPR compliance integrated into their, Apple, Google, Amazon and other app stores?

Big companies like Microsoft do offer GDPR guidance on how to make company IT compliant but the most important question is who in a company developing software, apps or websites should be the expert on GDPR compliance?

The simple answer is every employee involved in delivery must receive GDPR training but, logically the gatekeeper should be someone versed in compliance issues, how to verify and report on them. A smart company will ensure initial compliance by hiring an expert (possibly a consultant) but subsequently, the best person for the job is one of your testing / QA team.

A catastrophic mistake would be to have an employee brush up on GDPR by Googling it. Whoever the designated gatekeeper is should be sent on the appropriate certification course.

Getting back to MVP, it’s going to be up to founders, R&D heads, QA Managers to ensure that their processes evolve to ensure data protection is built-in from day 1. If this means an end to MVP and let the customer test it then this can only have a positive impact on customer satisfaction.

Personally, one of my bigger problems with GDPR is that the EU has repeatedly demonstrated a litigious attitude with anti-trust cases, often against companies like Apple, Google & Microsoft. Is GDPR just another EU kickstarter campaign?

Are you ready for GDPR or have you found a way to opt-out?

BUILD2018 Begins Today

BUILD2018 is upon us, what can we expect? BUILD is the annual Microsoft Developer Conference and whilst in past years, we have seen a lot of end-user oriented announcements in addition to the slew of  amazing APIs, SDKs, apps and improvements to existing Dev platforms, rumors expect BUILD2018 to be strongly developer oriented.

So what are expectations and predictions?

Anticipate a strong keynote by Microsoft CEO, Satya Nadella. He will no doubt highlight and reinforce the progress in realizing his vision for Microsoft. So far, this journey has delivered major increases in stock price, high customer satisfaction in consumer and business verticals and made Nadella one of Time Magazine’s 100 Most Influential in 2018.

In the wake of this success and the public debate following Facebook / Cambridge Analytica we can probably expect more in the keynote on developer responsibility for technology impact. No doubt we will also see presentations on social good thru technology and innovation, building on the success of Project Emma from BUILD2017:

WINDOWS 10: Whilst Microsoft has made it abundantly clear that cloud not desktop is the present and future focus, we are certainly going to hear about the outstanding Windows 10 Spring Creators Update that rolled out last week.

Expect to hear roll out metrics, major feature improvements and where we can expect Windows 10 to go from here. Expect to hear about Fluent Design, the new app UI. Which features of the OS have it, and which major apps are implementing it. Of course, it being BUILD2018 Developers should anticipate that Fluent Design, Universal Windows Apps & the Windows Store will all receive a strong developer focus. Microsoft is going to want you to build your apps with Fluent Design so be ready.

We can also expect to see Cortana mentioned, now not just on Windows 10 but fully cross-platform on mobile. I predict new devices, more app integrations (and integration tools), more services and a possible announcement of a changed UI / UX for Cortana coming soon. This is one more example of Microsoft continuing to build on their Windows on Mobile strategy.

OFFICE 2019: I predict a firm release date an even a demonstration of new version of Office at BUILD2018. If so, we can expect a focus on inking, more real-time collaboration and customization. In addition, the Microsoft Graph for Windows and Office will gain increased abilities. The star features will almost certainly be Slack killer, Microsoft Teams and improvements to the Office web apps.

Popular and growing services like IOT, BOTS, AI and Microsoft’s Cognitive Services should expect mention with improved SDKs and expanded abilities. Watch these demos because the Cognitive Services will blow your socks off.

Center stage will be Azure, Satya Nadella’s baby that effectively catapulted him into the CEO seat at Redmond. We anticipate continued expansion on services, partnerships and other improvements. This is where we expect to hear more about Containers and much on DevOps in Azure.

The big Azure question is, will we finally receive an Azure Serverless competitor for Google Firebase? Come on Microsoft, Azure Functions hasn’t gained the traction that Firebase has and there is a huge opening here with all the issues Firebase has cross-platform. BUILD2018 is a huge chance for Microsoft to onboard every last developer attending the conference or watching the livestream onto a robust, easy to integrate, serverless solution.

The other big money maker is Xbox.  What developer offerings will we see? Tough to speculate but expect a mention of the ongoing success of Xbox.

MIXED REALITY (XR): There is a chance we will hear something definitive about HOLOLENS 2, even if this is just a teaser. Expect more of a focus on OEM Windows 10 VR headsets and a presentation by Unity of what’s coming for Microsoft Mixed Reality.

Obviously, both Visual Studio and SQL will be getting a lot of attention in terms of new versions, new features and new integrations for all the platforms mentioned. Expect more improvements to cross-platform development on Xamarin.

I expect that we will get a hard launch date if not the launch of Visual Studio App Center, the replacement for HockeyApp which until now has been in Beta with several major features not implemented. BUILD2018 is the ideal opportunity to launch this to release and demo the full set of features here for “continuous everything – Build, test, deploy, engage, repeat …”

BUILD2018 - Visual Studio App Center 1

BUILD2018 - Visual Studio App Center 2

In short, this conference is going to deliver more excitement to the developers who work on these platforms. What’s missing from all this?

The Warptest POV

While I would like to bet that Microsoft are going to announce new hardware or new versions of existing hardware under the Surface brand, I’m not feeling it. I hope but don’t expect an announcement about any Surface Mobile device. Even with all the rumors of Andromeda, a folding ARM tablet/ phone that can run full Windows 10, it’s hard to believe that Microsoft are ready to drop this bombshell this week. This was the Warptest POV on Microsoft and their future in mobility.

The big announcements will be certainly Azure and Visual Studio focused but expect Progressive Web Apps to be mentioned more than once in the context of UWP. We can certainly expect more announcements focusing on open sourcing of development related technologies.

If I had to sum up my predictions, it would be that Microsoft are going to demonstrate a strong investment in building an even better Developer Community. Everything about BUILD2018 is going to be about driving more developers to invest in creating apps for the Windows and Office Stores but also to continue to force the competition to innovate at the same pace as Microsoft has been doing over the last few years.

Above is my latest video where I speak about all this. What do you expect or predict to hear this week?

Embracing the Obstacle

The obstacle. What is it?

GUEST POST: the following is a guest post by close friend Dan Shernicoff AKA @BrasSMan75. He’s written for Warptest in the past about WiFi Security and knows his stuff. Thanks for contributing Dan. Read on folks …

As a relatively freshly minted CSM I clearly remember the instructor saying that what Scrum does best is shine a light on the dysfunctions. It’s true, Scrum shows us where our problems are. Sometimes it shows us the boulders in our path and at other times we see the pebbles. What’s universal with these is they cost us in time, productivity, and job satisfaction.

Why We Test

When we write software, we test it to identify the bugs in our code. We try to find the success cases and the failure ones; we test both positive and negative outcomes. Having run all our test cases we invariably have found bugs – some are boulders that might require a redesign of the functionality while others are pebbles that we need to address to keep our users from failing while others yet are just gravel, annoying but of minimal consequence.

As we integrate Scrum we are constantly testing ourselves and our processes. Just as we found bugs in our code when we tested our software we will also find bugs in our processes, systems, workflows, and teams. Some of these are boulders that require major changes to how we get our work done while others are smaller and cause inefficiencies, distractions, and loss of productivity.

Just as we track our bugs in whatever tool we choose we use we need to track our obstacles. Just as we don’t take bugs personally – they’re mistakes we made while doing the best we could – we can’t take obstacles personally. We need to “embrace our obstacles,” give them a big hug, and show them the door. The only way that we can truly benefit from Scrum is not by marking the bumps in the road but by removing them.

obstacle - agile - tree trunk

The BrasSMan75 POV

I say all this with the self-knowledge that I have kicked myself many times for bugs that I have introduced into the code (and into the process at times.) I know this is hard but if we don’t do it we will never evolve our teams into their best versions. Whether the obstacle is that we run out of coffee the day before we resupply the break room every time (whether it is because we don’t order enough or we wait until we run out is irrelevant,) or that the tool that we have decided to use is unwieldy and not giving us what we need, or that we are having excessive meetings to communicate data that is easily accessible should we use the tools at our disposal, we need to address the problem. In the first case it’s simply a matter of changing the ordering habits, in the second we need to take time and try to find either ways to improve the tool or a tool to replace it with that will meet our need, while in the third we might need to train the people calling the meetings on how to use the tools to get the information.

As we develop and test our code we look for our bugs and fix them. As we go ahead with our stand ups, our retrospectives, and our sprints we need to look out for the obstacles and fix them. When we see the obstacle, we need to run up to it, give it a hug, and show it the door.

The Warptest POV

Dan raises a few interesting points here about our work habits, productivity and how we need to be engaged with the process for sprints to succeed. Agile can lead to rote adoption of the “rituals”. Framing the work in terms of the obstacle and how we behave when we encounter it is a valuable mindset. Thanks again Dan.

Augmented Reality Is Content…

Augmented Reality and by extension Virtual and Mixed Reality are the new content. Content is king. We’ve all heard it and read it multiple times over the last decade.

Content about us or our brand has an impact both positive and negative, often spreading virally.

Augmented Reality presents new challenges in how we manage and maintain our brand or reputation.

Ownership Issues

Augmented Reality is often location based. Just look at Pokemon Go, Snapchat / Instagram Location Based Stories and others. In the past, if content was written about you, your brand or physical site of your company, it was all about where it was hosted (blogs, social media, even video or Yelp reviews) but the big change with Augmented Reality is that the content is at your location and short of banning the use of apps or smartphones onsite, anyone can create AR content that is tied irrevocably to your location.

This slideshow requires JavaScript.

Museums and art galleries are experiencing a reality where anyone can add digital content to them. Imagine the scenario where someone either creates an inappropriate or unwanted AR. Control of the ether has flown the coop.

Some see this as a positive transformation, but the big question is, how do you curate this content? In fact, can a museum, gallery or any other private or public entity claim ownership of the location base for the content added as an AR?

Does a museum etc even have any direct redress opposite the app developers to have reputation damaging AR removed? Imagine a politician running a campaign and his opposition creating an AR at his campaign HQ or his next speaking venue. What if the location is controversial and Augmented Reality added by visitors contains strong or equally controversial opinions.

The idea behind Augmented Reality as an immersive, additional layer of rich data that offers an experience we wouldn’t otherwise get is a powerful one. AR apps are not just about social media but also useful in industry, education and can contribute to a museum experience. With ARKit and ARCore our smartphones make it easier for us to create and consume AR content but with great power, comes great responsibility. Or does it in this case? Is the genie simply out of the bottle?

The Warptest POV

Instagram as an example is something we can be calmer about. In paying homage to Snapchat Stories, Instagram made this content temporary so the damage of a negative or unwanted AR is limited. Prior posts have raised the idea that technological disruption can swiftly become our Frankenstein’s Monster. I’m sure if you read the (example) Snapchat terms and conditions, there are clauses that indemnify the company and they clearly have guidelines for approving “lenses” and geo-filters. Snapchat are only one player in the AR market, Quis custodiet ipsod custodes?

Once again, legislation lags behind innovation. Can Augmented Reality content be considered intellectual property? Does ownership of the location supersede any right to create location base content? Is there even a way for a brand or person to easily monitor their AR reputation?

Museums and others like them will have to find solutions to this issue. This is a huge opportunity. An opportunity for the app creators to deliver a solution for searching locations for this Augmented Reality content. As for an appeal process for owners to claim their location and be able to request that apps take down hostile, offensive or other allegedly inappropriate content, search engines offer a similar solution so why not?

Third party developers may be able to develop a reputation management solution if these AR platforms offer an API that supports this.

Here’s a free suggestion for you Dennis, Swarm (formerly the app known as Foursquare) displays photos of check-in locations why not similarly offer AR for the check-in?

Augmented Reality - Swarm app

The real opportunity is for us to use location based Augmented Reality constructively.

We seem to have failed somewhat with Facebook, Twitter and other social networks, let’s not fail with Augmented Reality.

Automated QA Yada Yada Yada

Another year of recurring discussions online how the silver bullet of automated QA is killing manual QA. This ongoing trend has had several impacts on the market, not all of them positive.

The Impact of Automated QA

1. Craftmanship is dying not manual testing. Do you know one person who is a craftsman in anything or have they been replaced by mass industrialized process? This is progress you say?

Junior testers feel driven to dive head-first into automated QA before they have learnt the craft of testing. They see automation as the end product, not as a means to implementing the tests and methodology they never had a chance to learn about. What do they spend their time learning, how the product works under the hood or the intricacies of their chosen automated QA framework?

2. QA Managers are super-powered, high octane, ninjas but, at a high cost. If you are a QA Manager then you had best be a full-stack test professional, who can manage a team, function as mentor, be an oracle for the product technology, generate dashboards, manage DevOps / ALM tools but also dedicate yourself to hands-on implementation of automated QA yourself.

Why? Because QA professionals have sold to companies that it is possible and optimal that one person is capable of doing this without compromising quality (ironic much?) of management. QA Managers are being forced to sacrifice their strategic responsibilities for tactical operations for what? Burn rate or simply because this idea stands unopposed?

Analogous to this, when is the last time your Development Manager spent a meaningful portion of their work day writing code?

3. Don’t shoot the messenger. Did we forget that the essence of our job is to report our findings? Often we report defects, overall quality of a build but also we raise red flags. We provide preventive treatment for issues before they go to production but if our underlying methodology is flawed where does that leave us?

When the silver bullet doesn’t live up to expectations, the messenger is not always viewed favorably.

The Warptest POV

We frequently cite Michael Bolton on his philosophy regarding testing vs checking and where automated QA falls into this. Here is one of his tweets, yet do we really work this way, evangelize this philosophy to our companies? It doesn’t seem so.

In recent days they asked Elon Musk about delivery problems in his company Tesla and his answer, I built too much on automation capabilities. If Elon Musk can admit that automation is not the magical solution (read the tweet below) then isn’t it time we considered the same?

Elon Musk’s admission has much further reaching implications than simply admitting over-reliance on automation. Musk admits that the role of human workers in successful delivery is underrated.

How do we refactor our methodology to redeem the human role in testing and QA?

In a nutshell, the perceptual bias towards automated QA needs a stake driving thru its heart.

Whilst I have used existing terminology and semantics to avoid confusion (manual vs automated QA, testing, etc.) this is clearly a major reason why Michael Bolton goes to great lengths to use accurate and appropriate terminology (again see his tweet above and many other of his posts, tweets etc for more).

In fact, there are technologies that cannot be tested with Automated QA, the existing frameworks aren’t mature enough to provide a solution. We can create partial solutions but we run the risk of becoming over-enamored with the solution as a product and not executing our tests on the actual product.

If we can get past this misconception about how we test, then maybe we can get to meaningful discussion about how we reframe optimal use and understanding of automated QA.

Two ideas I discussed today on a Facebook group for Israeli testers (in Hebrew) were: –

  • Test planning and design is an umbrella that provides coverage for your tests:

automated QA - umbrella

  • Testing debt – testing should be agile but not just in the sense of testers in the agile team. Not just in the sense of testing as part of the sprint. Every tester, regardless of the tools they use must be aware of how they and their tests integrate into the testing process.

It’s 2018, are you ready to focus on the optimal and correct way to use automated QA?

Windows on Mobile Is an Interesting Concept But …

Windows on Mobile is not just a concept anymore. Microsoft’s Windows Phone continued to lose market share, thru benign indifference, continued poor marketing or a management decision to allow it to slowly decline to be ultimately be replaced by what?

Windows on Mobile - Bryan

Microsoft did not want to allow a vacuum to develop in the interim. Did Windows on Mobile became the go-to strategy by default?

What Is This Windows on Mobile of Which You Speak?

If Microsoft couldn’t get us on their devices, they were damn sure going to get us on their cloud, SAAS and apps. There was a vision, where any smartphone on any platform would be a Microsoft phone. Regardless if your phone ran Android or IOS, you would be using Office365, OneDrive, Skype, Edge and other Microsoft services for personal and business use. Your phone would sync seamlessly with your desktop / laptop device and even allow you to continue your browsing session from mobile on your desktop PC.

Your Contacts, Calendar and other critical work and play information would work with Microsoft services. All these instead of Google or Apple services. Windows on Mobile is platform agnostic, service based and dominantly Microsoft driven. All with the same user and login details that you use to access your PC.

The use case was for everyone, home users, education, business and more. Using familiar tools and services, no matter where you are and what device you are using. No learning curve, just connect devices to the Microsoft account and keep on working.

The attraction of being able to choose what services and apps are at the core of your device and be able to remove the redundant ones has big appeal to many users. There was even talk in the past of Android phones coming pre-installed with Microsoft services and apps and not the Google equivalents.

SPOILER ALERT: every time you buy an Android phone or tablet, the big winner? Surprise, It’s Microsoft.

A Forbes article from 2015 explains the details of how Microsoft holds a slew of patents for Android which is a golden goose in licensing fees.

There had been a rumor in 2016 that Microsoft was interested in purchasing Cyanogen, back before they shuttered and open sourced the code. The anticipated strategy was to provide an Android ROM with all your Microsoft apps and services bundled.

Still the business strategists at Redmond knew better than to have one plan. Even before this, Microsoft has become the uncrowned king of cross-platform apps. Jump forward to 2018 and your Android smartphone’s best launcher is made by Microsoft, you can use Edge on Android which works with Cortana. All this and more.

Windows on Mobile - All the Android Microsoft apps

The Warptest POV

The idea of Windows on Mobile is an interesting one. Especially if you are a COO or IT Manager and you want your employees to have access to your Microsoft services outside the office.

Most Android device owners chose the OS and their phone specifically for Google Apps and services and are skeptical about replacing Chrome with Edge, Google Apps with Office and so on.

There is certainly a substantial section of Android owners who are happy to have both Google and Microsoft services running on their phone as needed. When Microsoft and Nokia were getting into bed together, the Nokia X family of devices were floated as Android out-of-the-box with Microsoft services installed. Microsoft cancelled these phones 5 months after acquiring Nokia though.

The idea appeals to many ex-Windows Phone owners who opted to leave the platform for Android but want to retain some of their comfort zone and don’t want to move away from these services and apps.

The fact is that Microsoft doesn’t need Windows on Mobile except as a way of retaining and onboarding more users to their apps, services and cloud, regardless of OS.

In a nutshell, Microsoft is providing an answer for Android & IOS users who want a Windows on Mobile solution, whether exclusively or not. All this may just be a stopgap solution that allows Windows Mobile to die gracefully while Microsoft work on their rumored, disruptive folding mobile device that finally delivers a true Windows 10 Mobile experience.

The rumored Microsoft “Andromeda” folding mobile tablet

Where do you stand on Windows on Mobile? Are you ready for an Android smartphone devoid of Google services or are you rooting for Andromeda?

Either way, Windows on Mobile offers a competitive push to Google & Apple. What are they going to do with it?

Boston Dynamics Can Solve Autonomous Vehicles Biggest Problem

The Boston Dynamics robots are successors to Robby the Robot  and they are not cute and often just a bit worrying to see in action.


Visions of robot uprisings, SkyNet and Terminators aside, these robots are astounding and can fill many roles that will complement humans or protect them. The US Navy announced SAFFIR, their firefighting robot prototype in 2015. The idea being SAFFIR can go into enclosed, smoke filled spaces aboard ships and fight fires instead of risking sailors.

Big Dog or the LS3 was a DARPA robot designed to carry heavy loads into the field, accompanying soldiers or marines.

We have military applications of cutting-edge robotics and yes, these robots make a lot of people feel uncomfortable but, military technology (when not weaponized) often migrates into the civilian market. How can these robots play a meaningful role in civvy street?

What’s the problem?

Yesterday, I posted about the first autonomous vehicle related fatality to occur and how technological disruption without ethical exploration of impact could be our Frankenstein’s Monster. In a related Facebook discussion with friends it occurred to me that calling the phase of releasing these vehicles onto city streets for “testing”, albeit with human monitors is optimistic at best. That said, I have found it very hard to find details online about the end-to-end testing methodology employed. The one source I did find was the California DMV site regulations for testing autonomous vehicles.

Autonomous vehicle testing - Boston Dynamics

I came away with several big questions:

Testing questions - Boston Dynamics

Besides questions there are some assumptions:

Stages of testing: one can safely assume that software is unit tested by developers and then the autonomous integrated systems are tested through simulators. What other stages of testing are there prior to testing in the real-world?

What is being tested: One can assume that the comprehensive list of features that allow an autonomous vehicle to function and interact in real-time are under test.

Test-cases: the different scenarios tested will range from functional tests, thru load and stress of the system into emergency scenarios.

Testing success: what is the pass / fail criteria for approving an autonomous vehicle to be released for general, real-world use? One assumes the tolerance for error is almost zero.

The Warptest POV

Autonomous vehicles certainly fit the description of technological disruption and their impact on the real world can be wondrous or catastrophic. A lot of which, depends on the depth at which they are tested.

Whilst I am certain crash-test dummies were used as in any automotive testing, this does not deliver the level of testing that IMHO is needed. Boston Dynamics has the solution. The testing stage before real-world testing where human drivers in other cars, bikes, trucks and pedestrians are all involved would be to build a testing environment that replicates the real world and to mitigate the risk to human testers, use Boston Dynamics robots to be the test-data used to run the different test cases.

The test-cases would have to provide optimal coverage of every conceivable scenario, but that data is waiting to be analyzed and derived by a good data scientist. Every recorded traffic mishap, accident, crime or fatality is a test-case that needs testing and this can be done in a testing environment that can replicate all weather conditions (and other variables). Another layer of testing will have to be the behavioral algorithms that allow autonomous vehicles to make critical decisions. If a vehicle is placed in a no-win scenario where either a passenger or pedestrian is sure of being hurt or killed, does the vehicle respond as expected and what is expected behavior? Is it based on learning or something else?

The good news is Boston Robotics or someone like them can provide a critical facet to this testing so that spontaneous pedestrian actions can be tested without risk.

SkyNet not - Boston Dynamics

Image via YouTube: with thanks to Terminator: Judgement Day.

Instead of being creepy robots that make some think we are one step away from SkyNet, these robots can be our path to safer autonomous vehicles.

Let me know if you think Boston Dynamics can solve this, if these robots creep you out or if you are building your post-SkyNet bunker after seeing the videos above.

Technological Disruption Is Evolving Beyond Our Control.

Technological Disruption has been on my mind for several months and yesterday it came to boiling point.

I wrote about this first in this post.

Technological Disruption: Truth and Consequences

Working backwards from yesterday we can see that unfettered disruption without testing and validation of the ramifications in the wild have major consequences:

The first death of a pedestrian by autonomous vehicle occurred yesterday in the US and while law enforcement are claiming that the car may not be responsible. Do pedestrians know how to interact with autonomous vehicles in the wild? Simple answer: NO. Especially as these vehicles fundamentally look just the same as your everyday vehicles. Where are the Asimov 3 Laws of Robotics?

Technology disruption - robotsImage courtesy of Microsoft Paint3D – Remix 3D library

Facebook and Cambridge Analytica have made us all feel naked and vulnerable due to the massive exploitation of our collective social footprints, our digital identities being taken without knowledge and used to effectively socially engineer us all regarding the 2016 US Elections. Freemium apps mean just this, you are ceding control of your digital identity when you freely share things. Our mobile devices have evolved around this whole precept but we have strong expectations of how that data is used and the transparency behind its use.

Technological disruption - Facebook

A few months ago an acquaintance was talking about how Waze has changed their quiet street where kids can play into a dangerous bypass route that cars speed down, anytime the adjacent major roads are blocked. Speaking to Waze (Google) and their City officials hadn’t made dent in the issue at last check.

AirB&B & Uber along with other sharing economy startups have a huge issue with background checks for hosts, guests, drivers and passengers … and the list goes on, not even counting data breaches like Equifax.

The Warptest POV

So I’m asking again, how do these companies challenge themselves to explore or test the ethical ramifications of their disruptive products before and after they reach the real world? Or is the genie out of the bottle and we need to learn how to live with this modern-day Frankenstein’s Monster.

Video clip from YouTuber iiAFX
This takes a special kind of mindset able to see technology as a product, an ability to perceive its place in different cultures and environments and how it interacts with all kinds of people in different demographics. Ultimately it needs a fearless mindset, a willingness to ensure that as an agent of change, the change is positive and beneficial.

Reliance on legislators is not a solution, whilst the EU are counting down to the deadline for GDPR compliance, other countries will legislate different standards eventually but these will be based on a variety of factors, not least the ability of public servants and politicians to comprehend what technological disruption is, how it works and the impacts we can expect. We can reasonably assume that different countries laws might end up being contradictory or intentionally in conflict. Good luck with that.

Quis custodiet ipsos custodes – Who watches the Watchmen?

Facebook and others will need to police themselves better but ultimately freemium apps rely on us, our data, our willingness to exist in their virtual worlds. We are their fuel and their monetization.

I am not advocating like some a mass deletion of our Facebook profiles, remember the same company also owns your much loved Instagram and ubiquitous WhatsApp. Still Facebook, Twitter and others need to make this a priority. We are talking about a  fundamental comprehension that symbiosis beats mutually assured destruction.

Ironically, much of what is being suggested online to resolve these things is a symptom of the problem. Issues like this cannot be solved with a hashtag, 14 characters or a snarky meme going viral.

For years we have heard how important DRM Digital Rights Management is for music, movies etc. where is our personal DRM? Can we disrupt Technological Disruption without inhibiting innovation?

These disruptive companies need to be much more transparent about their practices but also about the ethical challenges their technologies face and how they are solving these issues.

So if you want to comment, share ideas on the subject or just chime in.. Feel free.

If you know of a company that has a Chief Ethics Officer or a team that tests the ethical impact of their products then for sure let me know.

One thing I promise, whatever you share.. I will only be using that data for good.

That’s Right Unity Is Not Dealing With This Critical Issue

Unity are one of the biggest, if not biggest 2D & 3D game engines on the market. The company supports every conceivable, mainstream platform from mobile, thru desktop, Web to Virtual / Mixed Reality headsets.

Unity logo

This is the company that Apple & Google speak to when they want to role out ARCore & ARKit. By the time WWDC or Google annual conference role around, there is a beta version of Unity waiting in the wings with support for these changes.

Unity - ARCore
Unity - ARKit

If you have used a mainstream VR/MR app on Oculus Rift, HoloLens or HTC Vive then the likelihood is, that it was built in the Unity Editor.

Over the last year, a lot has changed for the company, especially instead of major version roll-outs, each subversion being released into Beta isn’t just bug fixes and iterations. Instead, these Beta sub-versions contain major new features and Unity isn’t shy about being transparent with their roadmap.

The big news is that Unity are making major inroads into the movie industry. They teamed with Neil Blomkamp to create the amazing animations for his OATS Studios movie shorts.

Many pundits believe that Unity is preparing themselves for an IPO with these changes, several major new strategic alliances and changes to the company board.

What Does Unity Need To Fix?

Ultimately, Unity provides a software development platform in their Editor. Code is written in C# and subject to the platform you are building your game or app on, the code is converted (e.g. conversion to C++ for IOS via IL2CPP). Notwithstanding the inability of the Editor to provide Developers with a mechanism to select all the platforms needed and run a cross-platform build script; this is not the major failing.

Their Editor has built in facility for Unit Testing via NUnit as part of Unity Test Runner.

Unity - Test Runner

QA can preview the scenes in the Editor but, Unity does not support Automated UI Testing on devices.

Once you have built your apps, you have no way except hands and eyeballs testing (more commonly argued over as “manual testing”) to test whether your IOS / Android / WebGL app functions or delivers the UI as expected.

Anyone who works in QA knows that standard practice is to incorporate automated UI tests using frameworks like Selenium for web testing and or, Appium for mobile apps.

These frameworks rely on the ability to recognize and map UI elements and objects with the app UI but Unity apps are a Black Box as far as Selenium or Appium are concerned. If you can’t map the UI elements and objects, then you can’t script clicks, swipes, text inputs or other simulations of real user behaviors.

This leaves game and app makers with 3 alternatives:

Unity - Testing

Manual testing alone is labor intensive, time consuming and repetitive. Cost aside it depends on skilled testers with the ability to catch and report bugs.

Customer testing is an oxymoron and often a disaster waiting to happen and yet some companies have no issue releasing their applications to their customers after only catching the critical issues.

Crowdsource testing is a good interim solution where companies lacking the testing personnel and is done by paying a 3rd party crowdsource company to deliver the warm bodies needed to test on personal devices for what amounts to first-to-find bug bounties.

The Warptest POV

Over the last year, I along with one of my QA Engineers tested several so-called automated testing solutions for Unity apps. Most didn’t make it out of the starting gate. Others showed early promise but needed extensive investment in development and testing to be anything more than a proof-of-concept.

All you need to do is search Unity’s Community Forums to see this is in high demand. Many companies and Unity personnel I spoke to online were interested to hear what we had discovered but if Unity want their community of 2D / 3D game, application and movie animation makers to deliver robust, well-tested products then automated UI testing needs to happen.

Unity - GDC

Today at 6.30pm Pacific Time, GDC, the Game Developers Conference kicks off. Don’t disappoint me Unity.

UPDATE

This is Unity’s summary blog post of their keynote at GDC. Color me disappointed, no mention of automated UI testing. Now I get it, automated testing in VR is a big challenge but choosing between doing nothing and at least supporting web / mobile automated testing on device, the choice is simple. FWIW if I had to choose between Unity and a platform that supports automated testing, the choice would be simple.

This is me throwing down the gauntlet Unity.

LinkedIn Has Everything Going For It, Except …

Is livestreaming a major missing feature from LinkedIn?

LinkedIn - livestream - app

Until now, Microsoft has not ventured seriously into the livestreaming niche, allowing big guns like YouTube & Facebook to dominate unrestricted streaming and Twitter’s Periscope to compete over mobile.

Since Microsoft had no meaningful social platform of their own (until the acquisition of LinkedIn) and with the death-by-degrees of Windows 10 Mobile .. to be replaced by a new, unknown, no-deadline mobile strategy, it was hard for Microsoft to leverage the need for streaming.

Why Does LinkedIn Need Livestreaming?

LinkedIn has a fight on its hands as Facebook invades the job networking / search niche. Facebook doesn’t waste time, they leverage their tools for personal users, then pages and or groups.

LinkedIn - Facebook

You can be sure the full suite of Facebook tools will be available to job seekers or companies looking to find top flight candidates.

LinkedIn - fight night

What could be better than seeing how a candidate handles live engagement by streaming their professional interests. This is infinitely better than a static resume, (searchable) LinkedIn profile or seeing a candidate’s engagement in groups. Recruiters get to observe speech patterns, passion, body language and introversion levels.

Job seekers benefit by being able to leverage this tool to fully promote themselves. A smart candidate will build a strategy for ensuring what they livestream will be polished, professional and relevant.

The Warptest POV

Job hunting is about using every advantage to stand-out and be noticed for your skills, knowledge and personality. To show recruiters that you are a good fit for the role they are looking to fill. So why doesn’t LinkedIn have livestreaming to allow you to do just that?

Right now, Microsoft has the technology but needs to integrate it into LinkedIn. Maybe they don’t see the value or maybe this should be a Premium feature. Where is this technology? Gaming and Social. Microsoft acquired social livestreaming company, Beam in August of 2016 and it’s now ported to Xbox as MIXER. Mixer is live game-streaming for Xbox and a cross-platform app.

Meanwhile since I started writing this, Microsoft and LinkedIn announced the integration of Word as the Resume Assistant, as predicted in the Warptest POV here, back in June 2016.

Should Redmond decide to take the Mixer technology and port it to LinkedIn (or Office and several other Microsoft platforms or technologies), then imagine applying for a job in real-time by livestreaming. What else can they bring to the pot? Bundle this in Skype with social integration or integration with Office 365’s Stream and you have a serious killer app. Stream isn’t just your company YouTube, videos you upload provide searchable speech-to-text captioning and facial recognition of people in your video. The livestreams you record can be saved thru Stream into your OneDrive. Shared thru Stream and turned into searchable video experiences.

Microsoft has the technology, all they need to do is find a way to bring it together and LinkedIn will compete on an even playing field with Facebook. Are you ready to stream your next job application?