Big Data 101: Thinking beyond National security Agency

Federal government investments and initiatives have long shifted the needle on technology innovation and adoption. Almost every case study on government funded innovation has a mention of how internet has its genesis in defense department’s DARPA initiative.

Last week, American public woke up to the fact that NSA, CIA and other security agencies were gathering phone records of some/most/all phone-calls to and from the United states, setting off a large public debate, perhaps what the whistle blower Edward Snowden wanted in the first place. It was interesting to hear US Senators and Congressmen try to explain technical jargon like metadata and applications of big data to their constituents. About how the call records turned over to federal agencies were just metadata of calls and not actual content of calls.

Corporate IT executives and CIO’s have already begun to recognize the value that good data analysts can bring and this incident is only bringing renewed attention on the potential of big-data. An unintended consequence of this saga, perhaps the real silver lining here is for technologists. Now that the program is out in the open, I wonder if there will be argument to commercialize the “mechanics” to reverse-engineering some of the big data technologies being discussed. One can argue that similar technologies used by NSA and federal agencies to gather and analyze large volumes (“big data”) of metadata about telephone records, can be used by commercial organizations. Say to parse through large volumes of data required for in-silico research to speed drug discovery.

Big data spells big money, not just for corporations. Data Scientists are already among the hottest category of IT professionals in the market. Play this out against the immigration debate and the message to younger generation of technologists in America is clear: plan a career in data analytics and science!

Other Links of interest

• While it is news now, the media has been talking about it for a while. NSA data center front and center in debate over liberty, security and privacy (Fox news article in April 2013)
• NSA's Big Data Platform Faces Enterprise Test: Accumulo, the data storage software developed by the National Security Agency, has taken another step toward the enterprise market. Sqrrl, the startup launched by former NSA technologists to commercialize Accumulo, has teamed up with Apache Hadoop provider Hortonworks to combine their technologies.
• NSA Reveals Cloud Plans, May Open-Source Some of Its Software
• Hadoop is an Open Source Revolution: Federal Computer Week Interview "Hadoop, and a handful of open-source tools that complement it, has no equal when it comes to making gigantic and diverse datasets easily available for quick analysis using clusters of inexpensive computers"

Indian offshoring at an inflexion point, yet again?

I try and periodically catch up on trends in offshoring. A few interesting trends in Indian offshoring seem to be shaping up pointing to an inflexion point (or maybe not)

• Anticipating yet another debate on US immigration overhaul debate with a bated breath: Read between the lines: while analysts, political pundits and others debate the outcome, it will probably be close to status quo. H1 work visas are too sacred and lucrative a cash cow for the US IT business and government to do a fundamental tinkering on
• Senior Indian offshoring Executive, Phanish Murthy ousted, yet again for sexual misconduct: Jury is out on whether he is being framed (again?) or just happened to follow the footsteps of former American President Clinton
• Cognizant overtakes Wipro and Infosys to take the title of second largest Indian offshoring firm (whatever that means to most of us). Bragging rights aside, it makes industry watchers reflect on how dynamic (and precarious) even the mega Indian offshoring firms are. Read between the lines: while TCS, Cognizant, Infosys, Wipro battle for title of “largest,” other global giants – IBM, Accenture, HP et al –are growing really really big in offshoring.
• Infosys brings back cofounder and Godfather of Indian offshoring, Mr. Narayana Murthy. Read between the lines: It may seem similar to what Steve Jobs did at Apple where there was little hope of a turnaround. By bringing back Mr Murthy, is Infosys admitting it is really in dire straits and that there is a bankruptcy of next generation leadership? (Indian media seems to be lukewarm to the idea. Ref Economic Times Why Narayana Murthy's comeback will set Infosys back)

Where all does all this leave the clients of offshoring firms? Status quo for now? And what about the masses of young Indian techies and wannabe techies in Engineering schools? Economist magazine recently had an interesting briefing on "India’s demographic challenge" summarizing "will soon have fifth of the world’s orking age population. It urgently needs to provide them with better jobs". Does the same apply to Indian offshoring segment too?

Big data to visualize migration

Looking at my linkedin updates from my contacts, I came across this fascinating example of using Big-data posted by a former colleague, Jai Ganesh, on his blog. The blog on “Networks of Global Migration” makes for an interesting read, and is topical: The debate over immigration in the US is reaching a crescendo!

 

A few insights from the blog worth mulling over

• According to the International Organization for Migration there are about 214 million international migrants across the world (about 3% of the global population), which is a significant increase over the year 2000 number of 150 million.
• According to IOM, countries with a high percentage of migrants include Qatar (87%), United Arab Emirates (70%), Jordan (46%), Singapore (41%) and Saudi Arabia (28%) and countries with a low percentage of migrants include South Africa (3.7%), Slovakia (2.4%), Turkey (1.9%), Japan (1.7%), Nigeria (0.7%), Romania (0.6%), India (0.4%) and Indonesia (0.1%).
• Global fund remittances by migrants were $529 billion in 2012. Remittances sent by migrants to developing countries were estimated at $401 billion in 2012. According to the World Bank, the top recipients of officially recorded remittances in 2012 were India ($69 billion), China ($60 billion), the Philippines ($24 billion), and Mexico ($23 billion). Other large recipients are Nigeria, Egypt, Pakistan, Bangladesh, Vietnam and Lebanon.

 

Female Migration Inflow clusters


Male Migration Inflow clusters

Bottomline: If picture is worth a thousand words, use of Big-data to tell the story of migration can substitute for reams of reports or powerpoints!

Looking beyond the political debate on immigration and offshoring

Infosys, Wipro, TCS in a headline on offshoring visas is sure to attract attention in a highly charged immigration overhaul debate. Therefore, it is not surprising to see headlines like the recent one in Economic Times titled “TCS, Infosys and Wipro abusing H-1B visa system: Senator” The article like many recent ones starts by saying “Amidst Congressional debate on the comprehensive immigration reform, a top US Senator has accused big Indian IT companies - TCS, Infosys and Wipro - of abusing the H-1B visa system.
"There are some specific abuses of H-1B," Senator Richard Durbin, said during a Congressional hearing on immigration reform by the powerful Senate Judiciary Committee on Monday, during which the lawmakers discussed threadbare the H-1B visa issues.”

The immigration debate and the proposed bill is focused on ensuring that companies that bring in people on H1 visas also hire a good percentage of local Americans (citizen, residents). Rightfully so, one might argue. The argument is that visas should be issued to hi-tech companies that generate value to American economy – Microsoft, google and the like - and also happen to be based in America, with a majority of employees being American.  (ref: NYT blog “New American Bill Threatens Indian Outsourcing Companies”)

Offshoring firms, many of which are headquartered in India (Infosys, Wipro, TCS) or incorporated in Bermuda (Accenture) try to neutralize the argument by trying local hiring and promising future growth with “onsite, proximity development centers.” Offshoring firms have been trying hard to hire and retain the few Americans (native or naturalized) whom they happen to employ. They also try and go the extra length to retain employees who gain permanent residency (green cards). Of course, the business model is skewed towards cost arbitrage that comes from serving clients with an offshore cost base, not being onsite-heavy.  (Personal note, in my past work-life, I joined and enjoyed the offshoring wave for nearly eight years with an offshoring major, despite being a US permanent resident. In a sense, I was an exception to the norm.)

Hiring local is an argument offshoring firms cannot win, leading to a lot of fuzzy math. For instance, NYT blog had to post a correction "An earlier version of this post said that Indian companies have invested $820 million in the United States to set up offices closer to their American clients. The correct figure is $5.9 billion." Rather than trying to argue on the exact investment - 820 million or $5.9 billion - there may be a better argument to be made in the TCO and value of offshoring

Outsourcing has long been a win-win proposition for corporate America. Sourcing non-core functions, including IT development and maintenance activities helps companies free up capital and resources to invest in other core activities that help in revenue generation. Offshoring extends the benefits of sourcing by leveraging a larger pool of resources across the globe, aided by seamless communication and technology. The fact is that much of Corporate America relies on cost benefits of offshoring. And while Microsoft, google and other hi-tech companies benefit from "foreign born" talent imported on H1 visas, rest of the corporate world also benefits by retaining the services of offshoring firms who need to import talent from offshore locations to service clients onsite.  The Microsofts and Googles may certainly be leading the technology innovation curve, but most other companies, large and small also leverage Information Technology to run their business. The "development and maintenance" of such IT systems requires armies of skilled technologists.

Refocusing the visa debate on the basic need from Corporate America is not as tenuous as it sounds.  However most logic is sure to get muffled in the rhetoric of political arguments.

Blasts in Boston and Bangalore . Time to end the cycle of terror

Like most of us who live in the US, I was shocked by the recent blasts during Boston Marathon. Though we don’t live anywhere near Boston, the incident certainly touched a nerve.


And then early this morning, I got a jolt while checking my emails with my bed-tea. My dad, who lives in Bangalore, had sent a note saying everything was “oaky”. Reading a few sentences ahead I realized he was talking about the blast in Malleshwaram, Bangalore.

The market area in Malleshwaram also happened to be my favorite hangout during college days years ago. My parents continue to frequent Malleshwaram, given the convenience of shopping and proximity to dozens of ancient Hindu temples.

I could go on with the blog, but at this time would stop with a simple appeal: Like most digirati and global citizen, I feel it is time to end the cycle of senseless terror and return to more innocent times in Bangalore, Boston and everywhere!

No Surprise behind Infosys’ Big dip in stock price: Reading the tea leaves

Scanning the news headlines on offshoring giant Infosys’s big dip this morning, I was reminded of the old Phantom "jungle saying" from Phantom comics I used to read growing up: “Biggest tree makes most noise when it falls” Employing 155,000 people, with revenues of over $7.23 Billion, Infosys is certainly a large tree. Though it has by no means fallen, a dip of 20% for a public company in a day is no small matter.

It has been more than a year since I moved from Infosys to the buy side of offshoring, taking on the role of an Enterprise Architect for a multinational organization. I continue to watch for trends and developments in the sell side, primarily on the state of offshoring firms. I have also been reflecting on the sustainability of the offshoring business model that hinges on recruiting fresh talent (“resources”) and operating on ever thinning margins. Two recent books "That's I.T" by Ramesh Revuru, a former Infoscion (my review) and "Offshore: India's Services Juggernaut" (my review) by Basab Pradhan who heads Global Sales & Marketing at Infosys provide contrarian bottom-up and top-down views on the business of offshoring. Basab’s views analyze the business model and potential "opportunities" while Ramesh takes a hard look at some of the operational challenges. (hyperlinks are to my reviews)

Not so long ago, Infosys was THE bellweather for offshoring, enjoying larger margins and client bill-rates by promising consistent, quality delivery. "Infosys is considered a bellwether for India’s outsourcing industry, which has been hit by weakness in the world economy. The industry provides information technology and business support services to companies from Europe to the United States." – WashingtonPost

The internal focus would be on maintaining high operating margin on every project/program/contract while trying to increase footprint at existing client locations. The intent of moving towards new revenue models and consulting continues to be aspirational while the bread and butter still comes from vanilla application development and maintenance (a.k.a ADM). All this while other service firms continue to grow their share in an increasingly limited global offshoring pie.

Although I was surprised by the INFY news and market reaction to it this morning, I guess there is not too much to be surprised if one were already reading the tea leaves. Just a sampling of the recent theme in the media:

• Questions on Business Model: A few months ago, there was an interesting piece in Economic Times analyzing “Will the last promoter-CEO S D Shibulal succeed in creating a new Infosys?” The answer so far has been no. And if one reads the tealeaves on today’s stock market knockout taking 20% off the value of the firm, it is a resounding No!
• Rumbling on slowdown in hiring, a canary in the offshoring coalmine if you will. (Hiring activity in IT sector may be muted this year: Infosys - Ref Business Standard)

Bottomline: Infosys and other offshoring firms “trained” the market to aspire for a 15-20 percent growth rate which was certainly practical when they were smaller. As offshoring firms get larger and less nimbler, the rate of growth (obviously) is not sustainable. When Infosys “forecasts” revenue to grow between 6 percent and 10 percent for the fiscal year ending March 2014, it sends shock waves in the industry. Should it shock those reading tea leaves at all?

Mid-week musing on age and id

Every now and then WSJ runs topical pieces examining social trends. The article “White Hair, Wrinkles Aren't Valid ID at These Drinking Establishments” is one such fascinating piece.


I was surprised to read that there are states in the US that made “universal ID checks mandatory in package stores (not bars)” and in some locales, "If you're 90 years old in a wheelchair, you're checked"

Just the other day I was reflecting on how store clerks just look at me, and do an automatic override when checking out the bottle of wine or some beer I occasionally have in my shopping cart. Is it my stray gray hair or the way I carry myself I wondered. Especially since I would routinely be asked for an ID for such purchases, even a few years ago. I am surely not in my twenties and have started looking my age I guess. There again, if I were to be asked for an ID, I could echo the thoughts of Alex Marshall who is quoted saying “I love it. My ego is soothed.”

The article also says some states and locales are repealing the law on ID check for those who look over a certain age, say 40 years and older. "It didn't work the way we thought," says Bill Davis, Republican chairman of the public-policy committee in the Indiana House of Representatives. Scott Pelath, the Democratic minority leader, says, "All it did was set folks off. We forgot that there's a fine line between clever and stupid."

Well put!