Posts Tagged big data

Big Data and Analytics as Game Changing Technology

If you ever doubted that big data was going to become important, there should be no doubt anymore. Recent headlines from the past couple of weeks of the government capturing and analyzing massive amounts of daily phone call data should convince you.

That this report was shortly followed by more reports of the government tapping the big online data websites like Google, Yahoo, and such for even more data should alert you to three things:

1—There is a massive amount of data out there that can be collected and analyzed.

2—Companies are amassing incredible volumes of data in the normal course of serving people who readily and knowingly give their data to these organizations. (This blogger is one of those tens of million .)

3—The tools and capabilities are mature enough for someone to sort through that data and connect the dots to deliver meaningful insights.

Particularly with regard to the last point this blogger thought the industry was still five years away from generating meaningful results from that amount of data coming in at that velocity. Sure, marketers have been sorting and correlating large amounts of data for years, but it was mostly structured data and not at nearly this much. BTW, your blogger has been writing about big data for some time.

If the news reports weren’t enough it became clear at IBM Edge 2013, wrapping up is Las Vegas this week, that big data analytics is happening and companies and familiar companies are succeeding at it now. It also is clear that there is sufficient commercial off-the-shelf computing power from companies like IBM and others and analytics tools from a growing number of vendors to sort through massive amounts of data and make sense of it fast.

An interesting point came up in one of the many discussions at Edge 2013 touching on big data. Every person’s data footprint is as unique as a fingerprint or other bio-metrics. We all visit different websites and interact with social media and use our credit and debit cards in highly individual ways. Again, marketers have sensed this at some level for years, but they haven’t yet really honed it down to the actual individual on a mass scale, although there is no technical reason one couldn’t. You now can, in effect, market to a demographic of one.

A related conference is coming up Oct. 21-25 in Orlando, Fl., called Enterprise Systems 2013.  It will combine the System z and the Power System Technical University along with a new executive-focused Enterprise Systems event. It will include new announcements, peeks into trends and directions, over 500 expert technical sessions across 10 tracks, and a comprehensive solution center. This blogger has already put it on his calendar.

There was much more interesting information at Edge 2013, such as using data analytics and cognitive computing to protect IT systems.  Perimeter defense, anti-virus, and ID management are no longer sufficient. Stay tuned.


, , , , , , , , , , , ,

Leave a comment

Where Have All the Enterprise IT Hardware Vendors Gone?

Remember that song asking where all the flowers had gone? In a few years you might be asking the same of many of today’s enterprise hardware vendors.  The answer is important as you plan your data center 3-5 years out.  Where will you get your servers from and at what cost? Will you even need servers in your data center?  And what will they look like, maybe massive collections of ARM processors?

As reported in The Register (Amazon cloud threatens the entire IT ecosystem): Amazon’s cloud poses a major threat to most of the traditional IT ecosystem, a team of 25 Morgan Stanley analysts write in a report, Amazon Web Services: Making Waves in the IT Pond, that was released recently. The Morgan Stanley researchers cite Brocade, NetApp, QLogic, EMC and VMware as facing the greatest challenges from the growth of AWS. The threat takes the form of AWS’s exceeding low cost per virtual machine instance.

Beyond the price threat, the vendors are scrambling to respond to the challenges of cloud, mobile, and big data/analytics. Even Intel, the leading chip maker, just introduced the 4th generation Intel® Core™ processor family to address these challenges.  The new chip promises optimized experiences personalized for end-users’ specific needs and offers double the battery life and breakthrough graphics targeted to new low cost devices such as mobile tablets and all-in-one systems.

The Wall Street Journal online covered related ground from a different perspective when it wrote: PC makers unveiled a range of unconventional devices on the eve of Asia’s biggest computer trade show as they seek to revive (the) flagging industry and stay relevant amid stiff competition. Driven by the cloud and the explosion of mobile devices in a variety of forms the enterprise IT industry doesn’t seem to know what the next device should even be.

Readers once chastised this blogger for suggesting that their next PC might be a mobile phone. Then came smartphones, quickly followed by tablets. Today PC sales are dropping fast, according to IDC.

The next rev of your data center may be based on ARM processors (tiny, stingy with power, cheap, cool, and remarkably fast), essentially mobile phone chips. They could be ganged together in large quantities to deliver mainframe-like power, scalability, and reliability at a fraction of the cost.

IBM has shifted its focus and is targeting cloud computing, mobile, and big data/analytics, even directing its acquisitions toward these areas as witnessed by yesterday’s SoftLayer acquisition. HP, Oracle, most of the other vendors are pursuing variations of the same strategy.  Oracle, for example, acquired Tekelec, a smart device signaling company.

But as the Morgan Stanley analysts noted, it really is Amazon using its cloud scale to savage the traditional enterprise IT vendor hardware strategies and it is no secret why:

  • No upfront investment
  • Pay for Only What You Use (with a caveat or two)
  • Price Transparency
  • Faster Time to Market
  • Near-infinite Scalability and Global Reach

And the more AWS grows, the more its prices drop due to the efficiency of cloud scaling.  It is not clear how the enterprise IT vendors will respond.

What will your management say when they get a whiff of AWS pricing. An extra large, high memory SQL Server database instance lists for $0.74 per hour (check the fine print). What does your Oracle database cost you per hour running on your on-premise enterprise server? That’s what the traditional enterprise IT vendors are facing.

, , , , , , , , , , , , , , , , , , , ,

Leave a comment

Winning the Coming Talent War Mainframe Style

The next frontier in the ongoing talent war, according to McKinsey, will be deep analytics, a critical weapon required to probe big data in the competition underpinning new waves of productivity, growth, and innovation. Are you ready to compete and win in this technical talent war?

Similarly, Information Week contends that data expertise is called for to take advantage of data mining, text mining, forecasting, and machine learning techniques. As it turns out the mainframe is ideally is ideally positioned to win if you can attract the right talent.

Finding, hiring, and keeping good talent within the technology realm is the number one concern cited by 41% of senior executives, hiring managers, and team leaders responding to the latest Harris Allied Tech Hiring and Retention Survey. Retention of existing talent was the next biggest concern, cited by 19.1%.

This past fall, CA published the results of its latest mainframe survey that came to similar conclusions. It found three major trends on the current and future role of the mainframe:

  1. The mainframe is playing an increasingly strategic role in managing the evolving needs of the enterprise
  2. The mainframe as an enabler of innovation as big data and cloud computing transform the face of enterprise IT
  3. Demand for tech talent with cross-disciplinary skills to fill critical mainframe workforce needs in this new view of enterprise IT

Among the respondents to the CA survey, 76% of global respondents believe their organizations will face a shortage of mainframe skills in the future, yet almost all respondents, 98%, felt their organizations were moderately or highly prepared to ensure the continuity of their mainframe workforce. In contrast, only 8% indicated having great difficulty finding qualified mainframe talent while 61% reported having some difficulty in doing so.

The Harris survey was conducted in September and October 2012. Its message is clear: Don’t be fooled by the national unemployment figures, currently hovering above 8%.  “In the technology space in particular, concerns over the ability to attract game-changing talent has become institutional and are keeping all levels of management awake at night,” notes Harris Allied Managing Director Kathy Harris.

The reason, as suggested in recent IBM studies, is that success with critical new technologies around big data, analytics, cloud computing, social business, virtualization, and mobile increasingly are giving top performing organizations their competitive advantage. The lingering recession, however, has taken its toll; unless your data center has been charged to proactively keep up, it probably is saddled with 5-year old skills at best; 10-year old skills more likely.

The Harris study picked up on this. When asking respondents the primary reason they thought people left their organization, 20% said people left for more exciting job opportunities or the chance to get their hands on some hot new technology.

Some companies recognize the problem and belatedly are trying to get back into the tech talent race. As Harris found when asking about what companies are doing to attract this kind of top talent 38% said they now were offering great opportunities for career growth. Others, 28%, were offering opportunities for professional development to recruit top tech pros. A fewer number, 24.5%, were offering competitive compensation packages while fewer still, 9%, offering competitive benefits packages.

To retain the top tech talent they already had 33.6% were offering opportunities for professional development, the single most important strategy they leveraged to retain employees. Others, 24.5%, offered opportunities for career advancement while 23.6% offered competitive salaries. Still a few hoped a telecommuting option or competitive bonuses would do the trick.

Clearly mainframe shops, like IT in general, are facing a transition as Linux, Java, SOA, cloud computing, analytics, big data, mobile, and social play increasing roles in the organization and the mainframe gains the capabilities to play in all these arenas. Advanced mainframe skills like CICS are great but it’s just a start. You also need Rest, Hadoop, and a slew of mobile, cloud, and data management skill sets.  At the same time, hybrid systems and expert integrated systems like IBM PureSystems and zEnterprise/zBX give shops the ability to tap a broader array of tech talent while baking in much of the expertise required.

, , , , , , , , , , , , ,

Leave a comment

New Products Reduce Soaring Storage Costs

The latest EMC-sponsored IDC Digital Universe study projects that the digital universe will reach 40 zettabytes (ZB) by 2020, a 50-fold growth from the beginning of 2010!! Do you wonder why your storage budget keeps increasing? And the amount of data that requires protection—backup on some sort—is growing faster than the digital universe itself.  This clearly is not good for the organization’s storage budget.

Worse yet, from a budget standpoint, the investment on IT hardware, software, services, telecommunications and staff that could be considered the infrastructure of the digital universe will grow by 40% between 2012 and 2020. Investment in storage management, security, big data, and cloud computing will grow considerably faster.

Last July BottomlineIT partially addressed this issue with a piece of reducing your storage debt, here. Recent products from leading storage players promise to help you do it more easily.

Let’s start with EMC, whose most recent storage offering is the VMAX 40K Enterprise Storage System. Enterprise-class, it promises to deliver up to triple the performance and more than twice the usable capacity of any other offering in the Industry, at least that was the case seven months ago. But things change fast.

With the VMAX comes an enhanced storage tool that simplifies and streamlines storage management, enabling fewer administrators to handle more storage. EMC also brings a revamped storage tiering tool, making it easier to move data to less costly and lower performing storage when appropriate. This allows you to conserve your most costly storage for the data most urgently requiring it.

HP, which has been struggling in general through a number of self-inflicted wounds, continues to offer robust storage products. Recognizing that today’s storage challenges—vastly more data, different types of data, and more and different needs for the data—require new approaches HP revamped its Converged Storage architecture. According to an Evaluator Group study many organizations only use 30% of their physical disk capacity, effectively wasting the rest while forcing their admins to wrestle with multiple disparate storage products.

The newest HP storage products address this issue for midsize companies. They include the HP 3PAR StoreServ7000, which offers large enterprise-class storage availability and quality-of-service features at a midrange price point.  HP StoreAll, a scalable platform for object and file data access that provides a simplified environment for big data retention and cloud storage while reducing the need for additional administrators or hardware.  Finally, it introduced the HP StoreAll Express Query, a special data appliance that allows organizations to conduct search queries orders of magnitude faster than previous file system search methods. This expedites informed decision-making based on the most current data.

IBM revamped its storage line too for the same reasons.  Its sleekest offering, especially for midsize companies, is the Storwize V7000 Unified, which handles block and file storage.  It also comes as a blade for IBM’s hybrid (mixed platforms) PureSystems line, the Storwize Flex V7000. Either way it includes IBM’s Real-Time Compression (RtC).

RtC alone can save considerable money by reducing the amount of storage capacity an organization needs to buy, by delaying the need to acquire more storage as the business grows, and by speeding performance of storage-related functions. While other vendors offer compression, none can do what RtC does; it compresses active (production) data and with no impact on application performance. This is an unmatched and valuable achievement.

On top of that the V7000 applies built-in expertise to simplify storage management. It enables an administrator who is not skilled in storage to perform almost all storage tasks quickly, easily, and efficiently. Fewer lesser-skilled administrators can handle increasingly complex storage workloads and perform sophisticated storage tasks flawlessly.  This substantially reduces the large labor cost associated with storage.

NetApp also is addressing the same storage issues for midsize companies through its NetApp FAS3200 Series. With a new processor and memory architecture it promises up to 80% more performance, 100% more capacity, non-disruptive operations, and industry-leading storage efficiency.

Data keeps growing, and you can’t NOT store it. New storage products enable you to maximize storage utilization, optimize the business value from data, and minimize labor costs.

, , , , , , , , ,

Leave a comment

5 Paths to IT Business Value

Earlier this year, Constellation Research identified four personas of the next- generation CIO, a CIO charged to deliver business value, not just keep the systems running. The four personas are: Chief Infrastructure Officer, Chief Integration Officer, Chief Intelligence Officer, and Chief Innovation Officer. Each delivers value in its own way.

The researchers also identified the top technologies that would create business value going forward. Topping the list was mobile, followed by cloud and data analytics/big data.  Rounding out their top five list were unified communications/collaboration and software enablement.

A similar Gartner survey asked CFOs which areas of technology they would like to see more investment and effort.  The top CFO choices: information and financial analysis followed by information management and knowledge sharing.  These were not much different than what Constellation found.

This shouldn’t be surprising. “IT provides a new source of opportunities for significant cost efficiency and organizational effectiveness that every business should exploit and leverage,” says Harry Wallaesa, CEO, WGroup, an IT management consulting firm. He goes on to cite 5 paths CIOs and IT managers should follow in pursuit of business value from IT.

First up, Wallaesa notes, is the convergence of ITO and BPO.  Technology and business process, long closely associated, finally are coming together in fact. From a business standpoint this translates into the simplification and standardization of sourcing deals and deal structure.  Standardization of contract components reduces cost of negotiating the contract, fewer staff needed to run the services because of similar contract structures, and easier control and analysis of cost since the contracts are done in like fashion. The payback: lower cost and better control.

Next comes the commoditization of IT and business services.  In fact, IT should position itself as a valuable service on par with other business services such as marketing and finance. This also, Wallaesa adds, can result in standardization and lower cost. A potential downside of the commoditization, however, may be loss of control. If IT cannot retain control of what systems and IT processes are used and the users can acquire systems their own the result may be the higher cost and loss of standardization,” he warns. In addition, the fact that technology has become so accessible has another downside, he notes—business units may go directly to the market, bypassing IT and controls altogether.

Similarly, the consumerization of IT may be a cost and control concern for the CIO. BottomlineIT first covered it here over a year ago. As previously noted, workers are bringing in their own smartphones and tablets, often paying for it out of their own pocket and downloading apps as they see fit. If workers can buy and bring in whatever they want to perform their work there arise both cost and control issues. At a minimum, it drives up the cost of IT support and governance as well as raising potential compliance and security issues.  In addition, the CIO should be concerned if people are not using consistent data systems and sources with consistent tools, which can result in apples-to-oranges comparisons.

Big data and analytics, Wallaesa’s fourth path, has emerged as so important to the business that CEOs are asking about it. The need for analysis of external and internal data to support the company is central to the success of the organization from a C-level perspective. Most CEOs view this as a specific part of their value-add. The CIO, then, is charged with ensuring valid, verifiable data is available for analytics. But again, the race to big data analytics using widely available tools any worker can access may lead them to bypass IT’s validation and governance role.  The CIO wants to ensure that the data is used consistently and controlled from a security standpoint.  This will be a big challenge for the CIO.

The growing interest in data analytics, big data, and mobility are clear signs of the overall convergence of IT and business, Wallaesa’s fifth path to business value, and maybe the one with the biggest impact on the CIO. The CIO has long struggled to earn a place as a peer among C-level execs. Yet, for too long many businesses have kept IT at arm’s length. No longer; astute business managers now realize that proactive use of IT can advance critical business objectives in terms of revenue, growth, and other strategic metrics, not just system uptime. Overall, this is a good thing, Wallaesa observes, because the business is taking ownership and responsibility for the use of IT. The CIO’s life may never be the same.

, , , , , , , , ,

Leave a comment

Speed Time to Big Data with Appliances

Hadoop will be coming to enterprise data centers soon as the big data bandwagon picks up stream. Speed of deployment is crucial. How fast can you deploy Hadoop and deliver business value?

Big data refers to running analytics against large volumes of unstructured data of all sorts to get closer to the customer, combat fraud, mine new opportunities, and more. Published reports have companies spending $4.3 billion on big data technologies by the end of 2012. But big data begets more big data, triggering even more spending, estimated by Gartner to hit $34 billion for 2013 and over a 5-year period to reach as much as $232 billion.

Most enterprises deploy Hadoop on large farms of commodity Intel servers. But that doesn’t have to be the case. Any server capable of running Java and Linux can handle Hadoop. The mainframe, for instance, should make an ideal Hadoop host because of the sheer scalability of the machine. Same with IBM’s Power line or the big servers from Oracle/Sun and HP, including HP’s new top of the line Itanium server.

At its core, Hadoop is a Linux-based Java program and is usually deployed on x86-based systems. The Hadoop community has effectively disguised Hadoop to speed adoption by the mainstream IT community through tools like SQOOP, a tool for importing data from relational databases into Hadoop, and Hive, which enables you to query the data using a SQL-like language called HiveQL. Pig is a high-level platform for creating the MapReduce programs used with Hadoop. So any competent data center IT group could embark on Hadoop big data initiatives.

Big data analytics, however, doesn’t even require Hadoop.  Alternatives like Hortonworks Data Platform (HDP), MapR, IBM GPFS-SNC (Shared Nothing Cluster), Lustre, HPCC Systems, Backtype Storm (acquired by Twitter), and three from Microsoft (Azure Table, Project Daytona, LINQ) all promise big data analytics capabilities.

Appliances are shaping up as an increasingly popular way to get big data deployed fast. Appliances trade flexibility for speed and ease of deployment. By packaging hardware and software pre-configured and integrated they make it ready to run right out of the box. The appliance typically comes with built-in analytics software that effectively masks big data complexity.

For enterprise data centers, the three primary big data appliance players:

  • IBM—PureData, the newest member of its PureSystems family of expert systems. PureData is delivered as an appliance that promises to let organizations quickly analyze petabytes of data and then intelligently apply those insights in addressing business issues across their organization. The machines come as three workload-specific models optimized either for transactional, operational, and big data analytics.
  • Oracle—the Oracle Big Data Appliance is an engineered system optimized for acquiring, organizing, and loading unstructured data into Oracle Database 11g. It combines optimized hardware components with new software to deliver a big data solution. It incorporates Cloudera’s Apache Hadoop with Cloudera Manager. A set of connectors also are available to help with the integration of data.
  • EMC—the Greenplum modular data computing appliance includes Greenplum Database for structured data, Greenplum HD for unstructured data, and DIA Modules for Greenplum partner applications such as business intelligence (BI) and extract, transform, and load (ETL) applications configured into one appliance cluster via a high-speed, high-performance, low-latency interconnect.

 And there are more. HP offers HP AppSystem for Apache Hadoop, an enterprise-ready appliance that simplifies and speeds deployment while optimizing performance and analysis of extreme scale-out Hadoop workloads. NetApp offers an enterprise-class Hadoop appliance that may be the best bargain given NetApp’s inclusive storage pricing approach.

As much as enterprise data centers loathe deploying appliances, if you are under pressure to get on the big data bandwagon fast and start showing business value almost immediately appliances will be your best bet. And there are plenty to choose from.

, , , , , , , , , , , , ,

Leave a comment

IBM PowerLinux Changes the x86 Calculus

With Linux reportedly the fastest growing operating system in the work and 99% of Global 2000 enterprises intending to include open source software in their portfolios by 2015 (up from 73% in 2012), clearly open source Linux has long crossed the adoption chasm and achieved enterprise acceptance. This is an ideal time for IBM to introduce its new PowerLinux servers, which deliver the performance of IBM Power Systems at x86 prices.

The IBM PowerLinux introduction includes two Linux-only models, a 2-socket, 2U rack server (7R2) and a 2-socket compute node (p24L) intended to plug into IBM’s newly introduced PureFlex appliance. Each socket contains 8 POWER7 cores and 256GB of memory, and IBM has tuned the machines for key tasks, like Big Data.  BottomlineIT covered the PureSystems introduction previously here. PureFlex is a computing appliance that accepts a variety of nodes in the form of blades, including PowerLinux as a compute node.

IBM’s goal was to deliver PowerLinux at a cost of acquisition competitive with x86 systems.  For example, the PowerLinux system including POWER7, virtualization (PowerVM), and the Linux open source OS (either Red Hat or SUSE) costs $21,282 (USD). By comparison, a comparable Dell/Intel system (able to handle the same workloads and running VMware vSphere Enterprise 5 and Red Hat Enterprise Linux subscription and support comes in at $22,650 (USD). A comparable HP/Intel system running VMware vSphere Enterprise 5 and Red Hat subscription and support costs $24,838 (USD). Comparative cost data provided by IBM.  If you don’t want the PowerVM hypervisor, there is the KVM option.

Both the Dell and HP systems run 2-socket, 16-core 2.4GHz ER-2665 Sandy Bridge processors with32GB of memory, two 300GB SAS drives (15k), a 4x1GbE network controller, and a SAS, DVD, RAID storage controller. The PowerLinux system brings a faster 16-core processor (3.55 GHz) while matching the other specifications.

So, in terms of speeds, feeds, and cost, PowerLinux not only meets but exceeds the leading x86 systems for running virtualized Linux workloads. When you look at virtualization for PowerLinux compared to VMware vSphere 5.0, PowerLinux looks even better. PowerLinux handles 16 virtual CPUs per virtual machine vs. 8 for VMware and 4 CPU threads per core vs. 2 for VMware. Throw in the secure hardware-based hypervisor for PowerLinux (PowerVM) vs. VMware’s software-based hypervisor and the PowerLinux machine is the clear winner.

In terms of workloads, one beta user, the University of Hamburg (Germany), compared two PowerLinux machines with ten x86 Linux servers for a big file serving workload. The result: PowerLinux delivered a 50% performance gain and a 30% lower cost of virtual file server acquisition.

If you look at a Big Data analytics workload—another increasingly important workload—the PowerLinux server with 4 threads per core has an immediate advantage over Intel’s 2 threads per core. PowerLinux also can handle parallel file systems across multiple servers using HDFS or the highly optimized IBM GPFS. In short, PowerLinux servers can natively exploit massively parallel processing across Linux clusters, which is what you want for Big Data.

PowerLinux already has been highly tuned by IBM. Should you deploy it as a compute node in the PureFlex appliance, you get the added integration, optimization, and automated expertise (patterns) IBM has packed into that device too.

The appeal of the PowerLinux system is that IBM streamlined it to match or exceed x86 cost and performance objectives. It, indeed, can beat comparable x86 machines running Linux virtualized workloads and do it at less than x86/VMware prices.

, , , , , , , , , , ,