MIS Chaos Monkeys

DevOps Case Study: Netflix and the Chaos Monkey

After some discussion of Netflix and the Chaos Monkey on our DevOps blog, I thought I would offer some detail of how Chaos Monkey and the Simian Army works. It's a great case study, posted on April 30th by C. Aaron Cois, from the SEI Institute at CMU.  I did not think to discuss until it was brought up. Maybe next semester, we'll start with it.

Anyway, Netflix's streaming service is a large distributed system hosted on Amazon Web Services (AWS). Since there are so many components that have to work together to provide reliable video streams to customers across a wide range of devices, Netflix engineers needed to focus heavily on the quality attributes of reliability and robustness for both server- and client-side components. In short, they concluded that the only way to be comfortable handling failure is to constantly practice failing. To achieve the desired level of confidence and quality, in true DevOps style, Netflix engineers set about automating failure.

Basically, you may have noticed that while the software is impressively reliable, occasionally the available streams of videos change. Sometimes, the 'Recommended Picks' stream may not appear, for example. When this happens it is because the service in AWS that serves the 'Recommended Picks' data is down. However, your Netflix application doesn't crash, it doesn't throw any errors, and it doesn't suffer from any degradation in performance. Netflix software merely omits the stream, or displays an alternate stream, with no hindered experience to the user, thus, exhibiting ideal, elegant failure behavior.

To achieve this result, Netflix dramatically altered their engineering process by introducing a tool called Chaos Monkey, the first in a series of tools collectively known as the Netflix Simian Army. Chaos Monkey is basically a script that runs continually in all Netflix environments, causing chaos by randomly shutting down server instances. Thus, while writing code, Netflix developers are constantly operating in an environment of unreliable services and unexpected outages. This chaos not only gives developers a unique opportunity to test their software in unexpected failure conditions, but incentivizes them to build fault-tolerant systems to make their day-to-day job as developers less frustrating.

This is DevOps at its finest: altering the development process and using automation to set up a system where the behavioral economics favors producing a desirable level of software quality. In response to creating software in this type of environment, Netflix developers will design their systems to be modular, testable, and highly resilient against back-end service outages from the start.

Simple example of State Diagram

As with the Sequence Diagram, the same author provides a partial example of a state diagram.  I hope this video helps to explain the process without having to go into great detail on all parts.

Sequence Diagram Video

Here is a video from Udacity explaining the System Sequence Diagram in a way that may make more sense for how it works.   Feel free to ask any questions.

Bitcoin Basics

I know we talked about Blockchain, but the question is always asked about why bitcoin prices fluctuate so much.  I found this link helpful to understanding digital currency basics at Coinbase.  You can also ask a question there, if these links are not helpful.

DevOps: What is DORA and Why You Should Care

Here is an introduction to DORA, and Dr. Nicole Forsgren, who was a Ph.D. student in MIS here.

Gene Kim, Jez Humble and Dr. Nicole Forsgren launched a new company called DORA. DORA stands for DevOps Research and Assessment.

DORA and the individuals behind it have been providing a lot of the science and analysis behind the State of DevOps survey and report for a number of years now (I posted the 2016 State of DevOps report on D2L). Here is the 2017 State of DevOps report (with new measures).

But what is DORA really about?  What is the business model to generate revenue? According to DORA CEO, Dr. Nicole Forsgren, after developing the annual State of DevOps Survey for years, they now have something like 25,000 responses over the past years. DORA has been able to create baselines and models from which they can compare how your organization compares to others who have taken the survey. They can pinpoint where you are lacking or not performing up to par, as well as where you are over performing. The entire process is built on rock solid statistical modeling and has already proven itself with several large enterprises.

If A Robot Offers You A Cookie…

I heard this on Science Friday last week, based on a new book, Soonish: Ten Emerging Technologies That’ll Improve and/or Ruin Everything by Kelly Weinersmith and Zach Weinersmith.  Read an excerpt on Amazon.

Here is a link to the podcast of the interview (or link to Soundcloud).

There is also a Soonish AR app.

This app is intended as a simple way to show you how AR works without you having to spend any money. Just open the app, point your camera at an image of the Soonish cover, and enjoy a custom space elevator Kyle Horseman designed for us (not to scale). 

Please note: You *do not* need to buy the book to use the app. We'd like if you did, but all you need to make it work is the free app and an image of the cover, like the one provided below.

Take a look... or listen.

Blockchain

At the talk during the IT Summit, of which many of you participated, I thought I would share with you other events and videos.  A video and articles helps to get your head around this process.

First, here is a brief 2-minute intro to Blockchain.

 

The speaker mentioned the MIT Media Lab.  Of course, there are many labs around the world working on this.  With that said, here is a link to an event on "The Business of Blockchain" at the MIT Media Lab and published (with videos) in the MIT Technology Review.  We can continue the discussion in class, but feel free to comment here, or ask questions.

The Essence of Agile Software

According to Martin Fowler,

It's been over a decade since the developers of agile methods first started to talk about their approaches. In this time agile thinking has changed from a niche activity to an approach that is widely used. However, like any popular technique, agile software development has suffered from semantic diffusion, so much of what we see under the name of agile doesn't bear much resemblance to what the early pioneers were doing. So I think it's important to revisit the essential elements of agile thinking

I've always seen the essence of agile thinking resting on two contrasts with traditional plan-driven software engineering.

Plan-driven engineering expects us to come up with a predictive plan that precedes development. The plan lays out the people, resources and timelines for the overall project. Software design is also done up-front, with implementation expected to conform with this design. Success is measured according to how well development follows this plan.

Agile plans are a baseline that we use to help us control change. Agile teams plan just as carefully as traditional teams, but the plans are constantly changing to reflect the things we learn during a project. Success is based on value delivered by the software.

Watch the 20-minute video of Martin Fowler explaining the history of software projects with the beginnings of plan-driven engineering (i.e., Waterfall) with agile plans.

Extra Credit: iSpace Events

Here is the general link to iSpace events, including Friday Tech Talks, Drop-In Help, and the Women Techmakers Hackathon, September 30–October 1.  Check it out @ http://new.library.arizona.edu/ispace.

More on VPN and End-to-End Encryption

VPNs
One of the most basic forms of encryption that you’re probably familiar with is the Virtual Private Network, or VPN.   There are a variety of different VPNs for different purposes, such as:

• Corporate VPNs that encrypt an entire network’s traffic
• VPNs to connect to corporate servers away from the office (ex. The UA VPN) 
• VPN routers that automatically reroute and encrypt users’ network traffic
• Web or application-based VPNs for personal use

Among the many VPN protocols, IPSec is considered the most secure. In fact, our friend Edward Snowden (or enemy and national traitor, depending on your point of view) has stated that the US Government has yet to crack IPSec, although they have in fact cracked other VPN encryption protocols. Below are a few free IPSec VPN options for personal use:

• CyberGhost

• TotalVPN

• Hotspot Shield (750MB per day limit, which is generous) 

End-to-End Encryption – Things to Know

Here are some examples that have end-to-end encryption technologies in place (with some caveats):

• WhatsApp: The company claims that it does not store messages on its servers, which means it can’t hand over messages if approached by law enforcement officials. 

• iMessage - Apple’s iMessages are end-to-end encrypted, which means they can only be read on users’ phones and the company can’t read them. But, you back up your messages in iCloud, then Apple can read them and could be forced to hand them over to authorities if provided with an appropriate warrant.

• Telegram: Telegram messages can be totally private if you want them to be. The company offers end-to-end encryption if users turn on the app’s “secret chat” feature and thus can’t read those user messages. Regular messages are stored on Telegram’s servers. 

• Signal: Owned by Open Whisper Systems, Signal is also end-to-end encrypted. The company explicitly states on its website that it “does not have access to the contents of any messages sent by Signal users.”

• Line: Line offers end-to-end encryption, but only if both the sender and recipient of a message turn on a feature called “Letter Sealing.” This will encrypt your messages so the company can’t read them, but regular messages without the feature are not end-to-end encrypted and Line may have to hand them over if required by Japanese law.

• Cyber Dust: Cyber Dust messages are encrypted end-to-end and the company claims they never even touch company servers. They’re also deleted from user phones as soon as they’re read (a la Snapchat). That means the company cannot hand over messages to authorities, even if a formal warrant was provided. “Once it’s gone it’s [gone],” CEO Ryan Ozonian told Re/code.

These Companies Can Read Your Messages

• Facebook (Messenger and Instagram):  Both Facebook Messenger and Facebook-owned Instagram encrypt messages only when they are en route between a user’s device and company servers where they are stored. This means Facebook might have to hand over private messages if required by law. 

• Google:  Messages sent via Google Hangouts are also encrypted en route and even on the company’s servers, but Google can still read them if needed. Encrypting the messages while on Google servers is intended to keep others from jacking in and reading them, but Google itself has the encryption key. This means Google might have to hand over private messages if required by law. 

• Snapchat:  Like Google, Snapchat messages are encrypted while at rest on Snapchat’s servers (though the company has the encryption key if needed). Snaps are deleted from the servers as soon as they’re opened by the intended recipients, and Snapchat claims  these delivered messages “typically cannot be retrieved from Snapchat’s servers by anyone, for any reason.” But unopened Snaps are kept on the servers for 30 days before being deleted. That means Snapchat might have to hand over unopened, private messages if required by law. 

• Twitter:  Direct messages on Twitter are not end-to-end encrypted. The company might have to hand over private messages if required by law. 

• Skype:   Microsoft-owned Skype does not offer end-to-end encryption for instant messages. They are stored on Skype’s servers for a “limited time,” which means Skype might have to hand over private messages if required by law.

• and soon, Comcast, Cox, and all IPSs...  (without an end-user agreement).  😱

Why End-to-End Encryption is So Important

Some of you may have read about the Wikileaks hack today (3/7).  But OF ALL THE revelations to come out of the 9,000-page data dump of CIA hacking tools, one of the most explosive is the possibility that the spy agency can compromise Signal, WhatsApp, and other encrypted chat apps. If you use those apps, let’s be perfectly clear: Nothing in the WikiLeaks docs says the CIA can do that.

Here is a great video explaining why end-to-end is so important these days.  (Click on the link; after ad is a video explaining end-to-end).  It's rather predictive of things to come.  But some important end-to-end tools - some of which you may already know about, but others may want you to consider.

Worried About Your Privacy Now? Here’s How to Protect It

Which leads to another recent and interesting piece of news related to encryption the media is using.  "How The Media Are Using Encryption Tools To Collect Anonymous Tips".
Like the video (above - please watch it), the article points out that there there is a renewed interest in leaks since Donald Trump's surprise election victory last fall, and a growth in the use of end-to-end encryption technology, have led news organizations across the country to highlight the multiple high-tech ways you can now send them anonymous tips.

The Washington Post, The New York Times and ProPublica have launched webpages outlining all the ways you can leak to them. ProPublica highlights three high-tech options on its page (in addition tthe Postal Service): the encrypted messaging app Signal, an encrypted email program called PGP (or GPG) and an anonymous file sharing system for desktop computers called SecureDrop. The Washington Post goes even further, highlighting six digital options.  Check this out!

Commenting is encouraged.  If you have nothing to say about this, you probably don't care who can hack your phone, or your laptop, or your text messages.  Just keep doing what you're doing.  Why should you worry?  You've done nothing wrong.

IoT Security

There are a number of good articles on this topic.  For example, Tech Crunch posted an article on "Why IoT Security Is So Critical".   "With the advent of IPv6 and the wide deployment of Wi-Fi networks, IoT is growing at a dangerously fast pace, and researchers estimate that by 2020, the number of active wireless connected devices will exceed 40 billion."

There is also the IoT Security Foundation, with links on news and encryption.  From there, there is a link to the 2016 IoT Security Foundation Conference. It includes videos on “Why is IoT Security so Hard and What Can We Do About It? and “Protecting the Connected Car". Great videos for those interested in this topic.

If you are working in this area, or just interested in IoT security in general, check out these links.

Music Licensing

A few of you are looking at music streaming, but some are looking into who is making money in this business.  I found this link on Public Knowledge (PK).  Their mission is to promote freedom of expression, an open internet, and access to affordable communications tools and creative works. They work to shape policy on behalf of the public interest.  PK works at the intersection of copyright, telecommunications, and Internet law, at a time when these fields are converging. PK's experience in all three areas puts it in an ideal position to advocate for policies that serve the public interest. They have a number of focus issues, including broadband privacy, net neutrality, copyright reform, and music licensing (to name but a few).

What's interesting about music licensing, is that you really can't listen to music streaming on most devices without it, but it is in dire need of improvement.  Briefly,

The complexity of music licensing is one of the main impediments to an optimal licensing system, both for compositions and for sound recordings. Another impediment is the impact of consolidation of big corporate rightsholders on the development of new platforms that give more options for independent artists and music fans alike.

PK believes the music distribution system should serve musicians and listeners. 

• Every company in the middle--from record labels to online services--exists to serve those two groups.
• The music licensing system should encourage a competitive, innovative market of new services that are accountable to music fans and musicians. 
• The largest content owners (major labels and publishers) should not take advantage of the current music licensing system to stifle competition and entrench their own gatekeeper positions.

Thus, PK is working for a fairer music marketplace where:

• artists can get their music on the market and receive a fair price for it,
• users can experience the music they want how they want, and
• new services can innovate without being beholden to gatekeepers.

There are some great links on this site.   To learn more check out the following:

• PK submitted these comments to the U.S. Copyright Office about the music licensing marketplace.
• Jodie Griffen (staff attorney, PK) testified for the House Judiciary Committee on the topic of music licensing.
• PK also produced a Copyright Tutorial for Musicians.

What has been your experience with music licensing?  Any comments?

Elon Musk's Hyperloop

Meet the Students Racing to Make Elon Musk’s Hyperloop a Reality.   Saw this article on Wired, and know that some of you are working on this topic.  This is a great article about a global competition, and links about how other countries are getting involved.  Here is a brief description:

If you (somehow) haven’t heard, hyperloop is a proposed long-distance, high-speed transportation system, Tesla and SpaceX CEO Elon Musk mentioned in 2012. Cargo or passengers would ride in pods (size TBD) that run through large tubes (above or underground) in something approaching a vacuum. With minimal air resistance, the pods would levitate above the floor of the tube with almost no friction. Propelled up to 700 mph or more, they could cover the distance between Los Angeles and San Francisco in just 30 minutes—a tantalizing alternative to an expensive flight or day-long slog through highway traffic. You can read way more about how the system works and the competing efforts to make it happen right here.

The articles goes on to explain the competition. In 2015, through SpaceX, Musk launched a global competition asking mainly student teams to give it a whirl. The mandate is to build a practical, safe, scalable, pod. The designs are judged for safety, innovation, and construction, but most really covet the prize for highest speed reached (with the caveat that the pod safely decelerates, too).

“What this was intended to do is encourage innovation in transportation technology,” Musk said on race day. “To get people to do things in a way that isn’t just a repeat of the past.”  Read more to find out what the global teams are doing.  Here is a video of some results:

Did Putin Direct Russian Hacking?

And Other Big Questions. This article in The Atlantic, dated Jan. 6, after a “declassified version of a highly classified assessment” was released, where the U.S. intelligence community laid out its judgment that “Vladimir Putin ordered an influence campaign in 2016 aimed at the US presidential election,” with the specific goal of harming Hillary Clinton’s “electability and potential presidency.” The report went on: “We further assess Putin and the Russian Government developed a clear preference for President-elect Trump.”

Who else has been hacked?
According to the article, Thomas Rid, writing in Esquire in October, noted that Russia began hacking the U.S. as early as 1996, five years after the demise of the Soviet Union, and added that the DNC hack concealed an even bigger prize for the Russians: the National Security Agency, whose secret files were dumped this August on Github and other file-sharing sites.

Then there is Germany. In May, BfV, Germany’s domestic intelligence agency, said hackers linked to the Russian government had targeted Chancellor Angela Merkel’s Christian Democratic Union party, as well as German state computers. Read the article for more details.

John Oliver posted about this at length recently, and the question of whether Putin directed Russian hacking is a topic that one student is posing for her tech briefing.  I post it here for your viewing pleasure.

Can Airbnb and Lyft Finally Get Americans to Trust Each Other?

But can we trust them?

One student is investigating racial bias in Airbnb.  Here is a Dec., 2016 HBR paper by Ray Firman and Michael Luca, both of Harvard University, on Fixing Discrimination in Online Marketplaces


It begins:

The first generation of online marketplaces, including eBay, Amazon, and Priceline, made it hard for sellers to discriminate. Transactions were conducted with relative anonymity. A user could negotiate a purchase without providing any identifying information until the seller had agreed to the deal. As a New Yorker cartoon famously put it, “On the Internet, nobody knows you’re a dog.” 

Except that platforms—and now their users—do know whether you’re black or white, male or female, human or canine [my emphasis]. And the internet has recently been revealed as a source of discrimination, not an end to it: With their identities uncovered, disadvantaged groups face many of the same challenges they have long confronted in the off-line world, sometimes made worse by a lack of regulation, the salience photos give to race and gender, and the fact that would-be discriminators can act without ever personally confronting their victims.

What happened, and what can we do about it?  Read the article to find out about smarter market design principles to consider.

---

Another article written two years earlier exclaims, How Airbnb and Lyft Finally Got Americans to Trust Each Other.  It argues that he sharing economy has come on so quickly and powerfully that regulators and economists are still grappling to understand its impact (see article above). But one consequence is already clear: Many of these companies have us engaging in behaviors that would have seemed unthinkably foolhardy as recently as five years ago.

We are hopping into strangers’ cars (Lyft, Sidecar, Uber), welcoming them into our spare rooms (Airbnb), dropping our dogs off at their houses (DogVacay, Rover), and eating food in their dining rooms (Feastly). We are letting them rent our cars (RelayRides, Getaround), our boats (Boatbound), our houses (HomeAway), and our power tools (Zilok). We are entrusting complete strangers with our most valuable possessions, our personal experiences—and our very lives. In the process, we are entering a new era of Internet-enabled intimacy.

Do you agree or is this unregulated market only useful for those who are allowed in?

IoT @ Microsoft

I think this is an easier way to share the blogs I post on our class blog (though if you read them, there are great articles on autonomous cars, AI, and much more about IoT). Here is an example from Microsoft.  For example, here is an article on "How IoT and AI are transforming cars with intelligent mobility."

According to the article, buy a new car, and you’re really buying a datacenter on wheels. By 2025, 100 percent of new cars will be connected cars, up from 35 percent today. And by 2030, 15 percent of new cars will be autonomous — and all will send, receive and analyze vast amounts of data.

Here are links to the keynote speakers who recently attended DesignCon, 2017. You can click on their talks. There are also videos there, though I haven't looked around the site extensively.  The optimistic tone is where the industry is going is expected on a Microsoft IoT blog, but there are good resources you can check out here.

What You Will Do In Your Self-Driving Car?

For those students exploring the autonomous car subject, here is an interesting twist.  Discussions of autonomous vehicles have become so commonplace that by the time driverless cars are widely available, the public’s excitement may be long over. It is time to ask what might be the impacts of autonomous vehicles on business and society? And if driving is left to the robots, will we also be inventing a new ridership economy?

The Auto Insurance Center recently conducted a survey that asked that very question. While this is clearly not the first such study, and most assuredly will not be the last, the findings present some interesting data.  Here is the top ten list of activities that 2,000 drivers from around the world said they will do when they are freed from the wheel (the survey link has more than 10):

1. Read a book
2. Catch up with friends and family via phone
3. Get work done outside of the office
4. Watch a television show
5. Watch a movie
6. Eat
7. Play video games
8. Sleep
9. Engage in sexual activity
10. Pray

It's hard to imagine that our time will be so "free to do what we want," especially given the security challenges facing this industry.  The article does talk about the ridership economy.  Does any of this seem real?

Purpose of this blog

The idea behind this blog is to offer links and articles that may benefit some of you in your search for resources, or that adds more information posted by students.  Rather than clutter your tech briefings on the class blog, I wanted a place to inform you of tech news I found interesting.  You can add your comments here as well.