, , ,

Big Data, by any other name, would smell as sweet

Modern companies thrive by harnessing and interpreting data. The more data we have, and the more we focus on analyzing it, the better we can make decisions. Data about our customers, data about purchasing patterns, data about network throughput, data in server logs, data in sales receipts. When we crunch our internal data, and cross-reference it against external data sources, we get goodness. That’s what Big Data is all about.

Data crunching and data correlation isn’t new, of course. That’s what business intelligence is all about. Spotting trends and making predictions is what business analysts have been doing for 40 years or more. From weather forecasters to the World Bank, from particle physicists to political pollsters, all that’s new is that our technology has gotten better. Our hardware, our software and our algorithms are a lot better.

Admittedly, some political pollsters in the recent United States presidential election didn’t seem to have better data analytics. That’s another story for another day.

Is “Big Data” the best term for talking about data acquisition and predictive analytics using Hadoop, Map/Reduce, Cassandra, Avro, HBase, NoSQL databases and so-on? Maybe. Folks like Strata conference chair Edd Dumbill and TechCrunch editor Leena Rao think not.

Indeed, Rao suggests, “Let’s banish the term ‘big data’ with pivot, cloud and all the other meaningless buzzwords we have grown to hate.” She continues, “the term itself is outdated, and consists of an overly general set of words that don’t reflect what is actually happening now with data. It’s no longer about big data, it’s about what you can do with the data.”

Yes, “Big Data” is a fairly generic phrase, and our focus should rightfully be on benefits, not on the 1s and 0s themselves. However, the phrase neatly fronts a broad concept that plenty of people seem to understand very well, thank you very much. Language is a tool; if the phrase Big Data gets the job done, we’ll stick with it, both as a term to use in SD Times and as the name of our technical training conference focused on data acquisition, predictive analytics, etc., Big Data TechCon.

The name doesn’t matter. Big Data. Business Intelligence. Predictive Analytics. Decision Support. Whatever. What matters is that we’re doing it.

, ,

Write once run everywhere, version 2.0

ethan-evansIn 1996, according to the Wikipedia, Sun Microsystems promised, 

Java’s write-once-run-everywhere capability along with its easy accessibility have propelled the software and Internet communities to embrace it as the de facto standard for writing applications for complex networks

That was version 1.0. Version 2.0 of the write-once-run-everywhere promise goes to HTML5. There are four real challenges with pure HTML5 apps, though, especially on mobile devices:

  • The specification isn’t finished, and devices and browsers don’t always support the full draft spec.
  • Run-time performance can be slow, especially on older mobile devices – and HTML5 apps developers can’t always manage or predict client performance.
  • Network latency can adversely affect the user experience, especially compared to native apps.
  • HTML5 apps can’t always access native device features – and what they can access may depend on the client operating system, browser design and sandbox constraints.

What should you do about it? According to Ethan Evans, Director of App Developer Services at Amazon.com, the answer is to build hybrid apps that combine HTML5 with native code.

In his keynote address at AnDevCon earlier this month, Evans said that there are three essential elements to building hybrid apps. First, architecting the correct division between native code and HTML5 code. Second, make sure the native code is blinding fast. Third, make sure the HTML5/JavaScript is blinding fast.

Performance is the key to giving a good user experience, he said, with the goal that a native app and a hybrid apps should be indistinguishable. That’s not easy, especially on older devices with underpowered CPUs and GPUs, small amounts of memory, and of course, poor support for HTML5 in the stack.

“Old versions of Android live forever,” Evans said, along with old versions of Webkit. Hardware acceleration varies wildly, as does the browser’s use of hardware acceleration. A real problem is flinging – that is, rapidly trying to scroll data that’s being fed from the Internet. Native code can handle that well; HTML5 can fall flat.

Thus, Evans said, you need to go native. His heuristic is:

  • HTML5 is good for parts of the user experience that involve relatively low interactivity. For example, text and static display, video playback, showing basic online content, handling basic actions like payment portals.
  • HTML5 is less good when there is more user interactivity. For example, scrolling, complex physics that use native APIs, multiple concurrent sounds, sustained high frame rates, multi-touch or gesture recognition.
  • HTML5 is also a challenge when you need access to hardware features or other applications on the device, such as the camera, calendar or contacts.
  • Cross-platform HTML5 is difficult to optimize to different CPUs, GPUs, operating systems versions, or even to accommodate single-core vs. multi-core devices.
  • Native code, by contrast, is good at handling the performance issues, assuming that you can build and test on all the key platforms. That means that you’ll have to port.
  • With HTML5, code updates are handled on the server. When building native apps, code updates will require apps upgrades. That’s fast and easy on Android, but slow and hard on iOS due to Apple’s review process.
  • Building a good user interface is relatively easy using HTML5 and CSS, but is harder using native code. Testing that user interface is much harder with native code due to the variations you will encounter.

Bottom line, says Amazon’s Ethan Evans: HTML5 + CSS + JavaScript + Native = Good.

, , ,

The API as an overloaded operator

Once upon a time, application programming interfaces were hooks that applications used to tap into operating system services. Want to open a port? Call an API. Need to find a printer? Call an API. Open a winder? Call an API. Write to a file? Call an API.

Developers still use classic APIs of course. They are necessary for both native and managed code. Windows, iOS, Android, Unix, Linux, all are stuffed to the brim with hundreds and thousands of APIs. In fact, one of the most useful features of an integrated development environment like Visual Studio, Eclipse and Xcode is to provide an handy reference to APIs, check their syntax and arguments, and help fill them out with autocomplete.

Classic APIs are fundamental. Cloud-based APIs, which provide loosely coupled function calls to services over the Internet, are more sexy and more dangerous.

The December issue of SD Times contains a feature by Alexa Weber Morales, “Connecting the World with APIs.” She explains that the variety of cloud-based APIs far exceeds the biggest, most visible examples, such as those from Amazon and Google. APIs are everywhere, from social media players like Facebook and Twitter, to business services like MailChimp and Salesforce.com.

Like electricity from the wall socket, or water from the kitchen faucet, it is easy to take cloud-based APIs for granted. Too easy. We outsource core functionality of our applications to cloud-based services, some free, some paid for by subscription. We expect them to work consistently. We expect them to be monolithic and unchanging. We expect them to be fast. We expect them to be secure.

We must not make any of those assumptions. Our software must be able to detect if a cloud-based API is offline or is running slowly, and should be able to handle such a situation gracefully. (I.e., not hang or crash.) We should never assume that APIs are secure and will keep our data safe or our customers’ data safe. We should not expect the API vendor to proactively notify us if they change some of the functionality within the APIs. It’s our job to be on top of any changes.

The availability of cloud-based APIs – unlike operating system APIs – is out of our hands. Our decision to upgrade a server’s OS is on our schedule, and we have time to read the documentation. When a mobile platform maker, like Apple, Google or Microsoft, releases a new operating system, we get plenty of notice and have plenty of time to understand about the newest APIs, the changed APIs and the deprecated APIs.

Not true with cloud-based APIs. While the three-letter acronym may be the same, our applications’ calls to a RESTful cloud-based APIs are not at all the same as our applications’ calls to native operating system services. While convenient, cloud-based APIs are ephemeral, distant and fundamentally unreliable. Never forget it.

, , , , , ,

Happy Thanksgiving

Tomorrow Americans will celebrate Thanksgiving. This is an odd holiday. It’s partly religious, but also partly secular, dating back to the English colonization of eastern North America. A recent tradition is for people to share what they are thankful for. In a lighthearted way, let me share some of my tech-related joys.

• I am thankful for PDF files. Websites that share documents in other formats (such as Microsoft Word) are kludgy, and document never looks quite right.

• I am thankful for native non-PDF files. Extracting content from PDF files to use in other applications is a time-consuming process that often requires significant post-processing.

• I am thankful that Hewlett-Packard is still in business – for now at least. It’s astonishing how HP bungles acquisition after acquisition after acquisition.

• I am thankful for consistent language specifications, such as C++, Java, HTML4 and JavaScript, which give us a fighting chance at cross-platform compatibility. A world with only proprietary languages would be horrible.

• I am thankful for HTML5 and CSS3, which solve many important problems for application development and deployment.

• I am thankful that most modern operating systems and applications can be updated via the Internet. No more floppies, CDs or DVDs.

• I am thankful that floppies are dead, dead, dead, dead, dead.

• I am thankful that Apple and Microsoft don’t force consumers to purchase applications for their latest desktop operating systems from their app stores. It’s my computer, and I should be able to run any bits that I want.

• I am thankful for Hadoop and its companion Apache projects like Avro, Cassandra, HBase and Pig, which in a only a couple of years became the de facto platform for Big Data and a must-know technology for developers.

• I am thankful that Linux exists as a compelling server operating system, as the foundation of Android, and as a driver of innovation.

• I am thankful for RAW photo image files and for Adobe Lightroom to process those RAW files.

• I am thankful for the Microsoft Surface, which is the most exciting new hardware platform since the Apple’s iPad and MacBook Air.

• I am thankful to still get a laugh by making the comment, “There’s an app for that!” in random non-tech-related conversations.

• I am thankful for the agile software movement, which has refocused our attention to efficiently creating excellent software, and which has created a new vocabulary for sharing best practices.

• I am thankful for RFID technology, especially as implemented in the East Coast’s E-Zpass and California’s FasTrak toll readers.

• I am thankful that despite the proliferation of e-book readers, technology books are still published on paper. E-books are great for novels and documents meant to be read linearly, but are not so great for learning a new language or studying a platform.

• I am thankful that nobody has figured out how to remotely hack into my car’s telematics systems yet – as far as I know.

• I am thankful for XKCD.

• I am thankful that Oracle seems to be committed to evolving Java and keeping it open.

• I am thankful for the wonderful work done by open-source communities like Apache, Eclipse and Mozilla.

• I am thankful that my Android phone uses an industry-standard Micro-USB connector.

• I am thankful for readers like you, who have made SD Times the leading news source in the software development community.

Happy Thanksgiving to you and yours.

, , ,

Echoing the echosystem

echoEchosystem. What a marvelous typo! An email from an analyst firm referred several times to a particular software development ecosystem, but in one of the instances, she misspelled “ecosystem” as “echosystem.” As a technology writer and analyst myself, that misspelling immediately set my mind racing. Echosystem. I love it.

An echosystem would be a type of meme. Not the silly graphics that show up on Twitter and Facebook, but more the type of meme envisioned by Richard Dawkins in his book, The Selfish Gene, where an idea or concept takes on a life of its own. In this case, the echosystem is where a meme is simply echoed, and is believed to be true simply because it is repeated so often. In particular, the echosystem would apply to ideas that are repeated around by analysts, technology writers and journalists, influential bloggers, and so-on.

In another time and place, what I’m now calling the echosystem would be called the bandwagon. I like the idea of a mashup between the bandwagon and the echo chamber being the echosystem.

We have lots of memes in the software development echosystem. For example, that the RIM BlackBerry is toast. Is the platform doomed? Maybe. But it’s become so casual, so matter-of-fact, for writers and analysts to refer to the BlackBerry as toast that repetition is creating its own truthiness (as Stephen Colbert would say).

Another is echosystem chatter that skeuomorphs are bad, and that Apple is behind the times (and falling behind Android and Windows 8) because its applications have fake leather textures and fake wooden bookshelves. Heck, I only learned about the term recently but repeating the chatter, wrote my own column about it last month, “Fake leather textures on your mobile apps: Good or bad?” True analysis? Maybe. Echoing the echosystem? Definitely

The echosystem anoints technologies or approaches, and then tears them down again. 

HTML5? The echosystem decided that this draft protocol was the ultimate portable platform, but then pounced when Facebook’s Mark Zuckerberg dissed his company’s efforts.

SOAP? The echosystem loved, loved, loved, loved, loved Simple Object Access Protocol and the WS* methods of implementing Web services, until the new narrative became that RESTful Web services were better. The SOAP bubble popped almost instantly when the meme “WS* is too complicated” spread everywhere.

Echoes in the echosystem pronounced judgment on Windows 8 long before it came out. Echoes weighed in on the future of Java before Oracle’s acquisition of Sun even closed and have chosen JavaScript as the ultimate programming language.

There is a lot of intelligence in the echosystem. Smart people hear what’s being said and repeat it and amplify it and repeat it some more. Sometimes pundits put a lot of thought into their echoes of popular. Sometimes pundits are merely hopping onto the bandwagon. The trick is to tell the differences.

, , , ,

Hurricane Sandy can’t stop the tech from Microsoft and Google

windows-phone-8It take a lot to push the U.S. elections off the television screen, but Hurricane Sandy managed the trick. We would like to express our sympathies to those affected by the storm – too many lives were lost, homes and property destroyed, businesses closed.

Microsoft and Google had scheduled tech events for the week of Oct. 29. Build took place as scheduled on the Microsoft campus in Redmond, Wash. Google cancelled its New York City launch event and offered its products rollouts via blog.

The big Microsoft news was the release of Windows Phone 8, with handsets from HTC, Nokia and Samsung set to go on sale starting in November. This follows, of course, the rollout of Windows 8 and the Surface with Windows RT ARM-based notebook/tablet device on Oct. 26.

Everyone that I know who has talked to who has used a prerelease Windows Phone 8 has been impressed. (I have a Windows Phone 7.5 device and find the Live Tile apps to be quite usable and exciting. I look forward to installing Windows Phone 7.8 on that device.) Through a strong program of incentives for app developers, there are many flagship apps for the phone already.

There are three compelling messages Windows Phone developers:

  • You can use Visual Studio and familiar tools to build apps for Windows Phone 8.
  • Windows Phone 8 is almost identical to Windows 8, so there’s minimal learning curve.
  • Windows Phone 8 is a reboot of the platform, which means you’ll face few competitors in the app store, called Windows Phone Store.

Of course, the downside is:

  • The installed base of Windows Phone 8 is nonexistent, compared to gazillions of iOS, Android and even BlackBerry OS.

If I were an entrepreneurial mobile app developer, I’d give Windows Phone 8 a try.

Google’s news was much more incremental: More hardware and a minor rev of Android.

The new hardware, announced in the Google Official Blog, is a new phone called the Nexus 4 and a 10-inch tablet called the Nexus 10. The big tablet has 2560×1600 display – that’s the same resolution as many 27-inch desktop monitors, and I’d love to see one.

Google’s seven-inch tablet announced during the summer, the Nexus 7, came only with 16GB of RAM and WiFi. Now you can get it with 32GB RAM or GSM-based cellular connections using the HSPA+ mobile standard. These are good hardware upgrades, but aren’t “stop the presses” material in the weeks surrounding the launch of Windows Phone, Windows Phone 8, Surface and Apple’s iPad Mini. Heck, the tablet doesn’t even have 4G.

The operating system update is Android 4.2, which is still called Jelly Bean. There are plenty of consumer features, such as a spherical panoramic camera mode, and a smarter predictive keyboard. The ability to support many users is a good feature, and one frankly that is long overdue for these expensive tablets.

Expect to see more about Android 4.2 at AnDevCon IV, coming up Dec. 4-7, 2012. Maybe someone will bring one of those 10-inch tablets so we can see the screen.

, , , ,

Apple’s victory over Samsung should drive innovation

The jury is in: Samsung was found to have infringed upon Apple’s numerous mobile patents. The jury’s verdict form, handed down in the United States District Court in San Jose, Calif., found that in many cases that the “Samsung entity has diluted any Apple trade dress(es).” What’s more, Apple proved “by a preponderance of the evidence that the Samsung entity’s direction was willful.”

Ouch. This is the worst case scenario for Samsung. Forget about the US$1.049 billion in damages that Samsung is supposed to pay Apple. What this means is that the jury agreed with what everyone knew simply by looking at the hardware and playing with the software: the Samsung Galaxy Tab 10.1 is just like the iPad.

On the short term, this ruling is going have a chilling effect not only on Apple, but on every maker of Android devices. The more similar the devices are to Apple’s iOS phones and tablets, the more scared the hardware manufacturers are going to be. (That is, if the verdict stands and isn’t overturned on appeal.)

We can expect to see a lot of introspection within the Android ecosystem. Google, Samsung and the other device manufacturers will look close, really close, to make sure they stay away from the specific patents cited in this case.

We can expect to see software updates and hardware guidelines that will take Android devices farther from Apple’s devices.

On the short term – this will depress sales of Android devices. On the longer term, we will see a ton of innovation that will truly differentiate Android from iOS.

For too long, Android handset- and tablet-makers have been trying to get as close to the iPhone and iPad design as possible. It’s not laziness or a lack of technical savvy, in my opinion. It’s just that Apple has done such a good job of defining the smartphone and tablet that consumers expect that, well, that’s just how the platforms should work.

Salespeople want to sell Android devices that are identical to Apple devices, only less expensive.

Consumers who choose Android are sometimes making those selections based on technical merit, but are sometimes looking for something that’s just like an iPhone/iPad, only different. Perhaps they want more memory, perhaps a bigger phone screen, perhaps a smaller tablet screen, perhaps a slide-out keyboard, sometimes a removable battery, sometimes simply a brand that isn’t spelled “Apple.”

Of course, with rumors that Apple is about to release a 7-inch iPad, the job of Android tablet companies is only going to get harder. In my own informal polling, folks who have purchased 7-inch tablets have done so mainly because Apple doesn’t sell one.

For the next year or so, Samsung and the whole Android community will fall back and retrench. That will involve unleashing innovation that may have been stifled, as they preferred to imitate the iOS designs instead of pushing their own ideas.

Imitation may be the most sincere form of flattery – but in the smartphone and tablet markets, imitation is off the table. For good.

, ,

Oracle, Sun, Winners, Losers

It looks like Oracle is going to buy Sun Microsystems for $5.6 billion (net of Sun’s cash cache). Maybe the deal won’t happen. Maybe IBM will swing in with a counter offer. At this point, though, the odds are good that Oracle’s going to end up owning Java and all the other Sun technologies.

Oracle is getting a lot of very nice intellectual property. Whether that IP — as well as Sun’s product lines, maintenance agreements, licenses, consulting gigs and sales contracts — are worth $5.6 billion, that’s hard to say.

Overall, though, Oracle is clearly the biggest winner in this deal. It’s getting core technology that will cement its position in the application server market, and also give it obvious control over key industry specifications like the Java language, the enterprise Java EE platform, and the very important Java ME platform. Expect Oracle to exercise that control.

Let’s see who else wins and loses.

Loser: IBM. For years, I’ve speculated that IBM would purchase Sun just to secure a tight control over Java – which is a core technology that IBM depends upon. Now, that technology, as well as the Java Community Process, is going to fall into enemy hands. Bummer, Big Blue.

Winner: Java. Java is very important to Sun. Expect a lot of investment — in the areas that are important to Oracle.

Loser: The Java Community Process. Oracle is not known for openness. Oracle is not known for embracing competitors, or for collaborating with them to create markets. Instead, Oracle is known to play hardball to dominate its markets.

Winner: Customers that pay for Sun’s enterprise software. Oracle will take good care of them, though naturally there will be some product consolidation. Software customers may like being taken of by a company that’s focused on software, not hardware.

Loser. Customers that use open-source or community-supported versions of Sun’s software. Oracle is not in the free software business, except when that free software supports its paid software business. Don’t expect that to change.

Winner: Enterprise Linux vendors. Red Hat and other enterprise Linux distros will be dancing if Oracle decides that it doesn’t want to be in the Solaris business. On the other hand, this purchase makes it less likely that Oracle will spend big dollars to buy Red Hat in the near future.

Loser: MySQL customers. If Oracle keeps MySQL, expect it to be at the bottom of the heap as a lead-in for upgrades to Oracle’s big-gun database products. If Oracle decides not to kill or spin off MySQL, that’s going to mean disruption for the community.

Winner: Eclipse Foundation. Buh-bye, NetBeans! Oracle is heavily invested in Eclipse, and would be unlikely to continue investing in NetBeans. It’s hard to imagine that anyone would buy it, and the community probably couldn’t thrive if Oracle set it free.

Loser: Sun’s hardware customers. If Oracle stays in the hardware business, expect those Sun boxes to be only a bit player in Oracle’s product portfolio. If Oracle sells it, whoever buys it will probably milk it. How does “IBM System s (SPARC)” sound to you? Not very attractive.

Biggest Winner: Sun’s shareholders, including employees with options. After watching their shares plummet in value, and after getting a scare from IBM’s paltry offer, they must be counting their blessings right now.