In his new book, The Connected Company, Dave Gray tells us how social media has changed the world of business. Customers are now reshaping the way companies work. Twenty years ago, when unhappy with a product or a service, we told 5 or 10 friends about our gripes. Today we are connecting, forming networked communities that allow us to rapidly share information and self-organize into powerful interest groups. With the help of Twitter, Facebook and others, we are able to share our bad or good experiences to tens of thousands people. These changes mean that business models have to be changed for companies to survive.
This interview was recorded as part of the Mind Tools Expert Interview podcast series. For more information, go to www.mindtools.com.
In this interview, Dave Gray talks about what is to be a “Connected Company.”
“The success of companies like Google, Facebook, Amazon, and Netflix, not to mention Wall Street firms and industries from manufacturing and retail to healthcare, is increasingly driven by better tools for extracting meaning from very large quantities of data. ‘Data Scientist’ is now the hottest job title in Silicon Valley.” – Tim O’Reilly
As pointed out by Tim, we are surrounded by Data Science, also known as big data. I have now found out that there is a new MSc course in Data Science taught at the University of Dundee. Mark Whitehorn, the founder of the course, was kind enough to answer some questions.
Hi, Mark, I hear you’ve just launched a new MSc course in Data Science at the University of Dundee. So, first questions first: what’s data science?
Very cool at the moment. It’s a term we’re hearing more and more these days, and, as with many new terms, different people define it in different ways, and those ways are evolving even as we speak. In a nutshell, a data scientist is someone with well-developed skills in analysing data, especially in analysing large amounts of data that does not fit readily into traditional tabular database structures. That kind of data is often called Big Data, another term much in the news.
Is it a full-time or a part-time course?
It can be either. You can take it as a full-time course of study lasting one year, or as a part-time one lasting two. Most of our current students take the part-time option because they are already in employment and the course is designed to accommodate this. Each year the part-timers attend two separate weeks of lectures in Dundee, plus about 2 extra days for examinations. So the total time they have to spend in Dundee is 12 days per year; the rest of the time is home-based study with assignments, phone tutorials and a project. Full-timers must be based in Dundee and attend four full weeks of lectures and also do the assignments, tutorials etc. Both part and full time students also do a research project.
Why now and why Dundee?
As well being a Professor at the University, I also work as a consultant and my commercial experience tells me that businesses are crying out for people who can extract useful information from huge data sets; information covering performance, trends, behaviours, predictions, any scrap of information that can help a business to stay ahead of the curve. With my commercial hat on I constantly see the need for people who are trained in data science all the time; so as an academic it made perfect sense to set up the course!
Furthermore in the School of Computing at the University of Dundee we have been running a very successful MSc course in Business Intelligence since 2010. It has been very well received by the students, about 75 of them so far, and all of our full-time graduates have found employment in the BI field. The part-timers already had jobs and many of them have moved to better jobs. Launching the Data Science course seemed a natural progression and has already proved popular: we had about three times more applicants than places available for the January 2013 intake.
Finally the research work we do at Dundee has involved big data and data science since about 2007; in other words, we aren’t just talking about it, we’ve been doing it for at least five years. We do actually know how to do it for real!
Anything else you’d like to say about data science?
I think it is complex to define fully but we have tried to give the flavour of the sorts of skills we expect our students to possess by the time they graduate.
General skills include:
- excellent analytical capabilities
- machine learning
- statistics, maths and data mining
- algorithm development, code writing
- data visualisation
- understanding multi-dimensional database design and implementation
Specific skills include:
Technologies to handle big data
- Hadoop and related technologies
- MapReduce and its implementation on differing software platforms
- NoSQL databases
Knowledge of languages such as
- SQL, MDX , R
- Functional and OOP languages such as Erlang and Java
General characteristics include:
- Interdisciplinary interests
- Excellent communication skills
- Insatiable curiosity
Insatiable curiosity – I like that!
Yes, and it’s really an excellent indicator of the sort of person who will excel as a data scientist. It denotes something rather like the old meaning of the hacker mentality. Not someone who breaks into systems but a hacker in the sense of someone driven to understand, to explore all the options, to try all the permutations. Someone who works on a problem and suddenly notices it is three in the morning, someone who has survived for days on flat food.
Duncan Ross, Director of Data Sciences at Teradata, has said that “The first and most important trait is curiosity. Insane curiosity. In many walks of life evolution selects against the kind of person who decides to find out what happens ‘if I push that button’. Data Science selects for it.” I think he’s absolutely right.
Any final thoughts before we wrap it up?
It has been great fun and very rewarding to run the BI course and I am sure the DS course will be equally so, not just for me and my colleagues, but also for the students. I’ll give the last word to some of those who have already graduated from the BI course:
“Enrolling on the Business Intelligence Master’s Degree at the University of Dundee has been the single most important thing I have done to further my career since starting to work in IT 18 years ago. Not only have I thoroughly enjoyed the course and project work but I have met some great people and established a strong network of friends who work in the industry. Not to mention landing a dream job as a Data Scientist with Teradata at the end of it!” Chris Hillman
“It was hard work at times, but the reward you get as always, is proportionate to what you put into it. And it was fun. Given the opportunity, I’d definitely do it again – two years of my time well invested, with no regrets and some great adventures on the way.” Jon Reade
“The style is never dogmatic and always open-minded allowing everyone to input their own ideas in this fluid area of the business and scientific world.” Gordon Meyer
“I feel it was a great decision to relocate from my home country for this amazing year that provided me sound knowledge and further inspiration.” István Poprócsi
“I was delighted when I finally completed the course and was awarded an MSc. with distinction – but at the same time felt rather sad that this great experience had come to an end. I’d learned so much and loved every minute of it.” Andy Hogg
Prof. Mark Whitehorn specializes in the areas of data science, analytics, business intelligence (BI) and Big Data.
On the academic side, Mark holds the Chair of Analytics at the University of Dundee where he designed and runs two Masters courses, one in Business Intelligence, the other in Data Science. He also works with the prestigious Lamond labs. applying data science to proteomics.
In addition Mark works as a consultant to national and international companies, designing analytical systems. He is also a well-recognized commentator on the computer world, publishing articles, white papers and 11 books on database and BI technology.
For relaxation he collects and restores old cars, which keeps him out of too much trouble. He only wears a tie under duress and unashamedly belongs to the beard-and-sandals school of computing. (And he doesn’t take life as seriously as this photo suggests!)
A little bit of history
Created in 2001 under the name of JavaPolis, Devoxx has always sold out weeks before the event. In November 2012, 3400 delegates gathered in Antwerp for the biggest vendor-independent Java Conference in the world. Devoxx takes place in the MetroPolis in Antwerp, Belgium. As a result, the conference benefits from the cinemas technical facilities (huge screens and THX audio setup). This annual conference is organized by Stephan Janssen and the Belgian Java User Group (BeJUG).
In 2012 the Paris Java User Group brought Devoxx to Paris. As a first edition, the number of delegates expected was between 600 and 900 – eventually 1200 came and numbers had to be turned down as the venue could not cater for more!
Two conferences in one year was not enough, so in 2012 we saw Devoxx 4 Kids organized in Ghent and Brussels attracting teenagers between 10 and 14 years. The teenagers played with Scratch, programmed Lego Mindstorms and discovered the wonderful world of Mars Rovers and the NAO robot.
Devoxx is coming to London
The good news is that 2013 will be even busier with the arrival of Devoxx UK. Yes, Devoxx is coming to London on March 26th and 27th at the Business Design Centre in Islington. Devoxx UK will be spearheaded by Martijn Verburg, Dan Hardiker, James McGivern, Ben Evans and Stephan Janssen in close collaboration with the London Java Community.
I have just read in the latest newsletter that the following invited speakers have already been confirmed:
- Charles Oliver Nutter (Red Hat) – Inventor of JRuby
- Gemma Silvers (Playfish) – Makes games for millions of users and an Enterprise CMS guru
- James Ward (Typesafe) – Expert on modern JVM languages and cloud deployment
- Kirk Pepperdine – Performance Tuning guru
- Dr Holly Cummins (IBM) – Wearable computer specialist
- Ola Bini (Thoughtworks) – Inventor of multiple JVM languages
- Jos Boumans (Krux) – Cloud/Scalability guru
- Stephen Chin (Oracle) – Nighthacking Tour Java evangelist
- Martijn Verburg (jClarity) – “The Diabolical Developer” and LJC leader St
- Stephen Colebourne (OpenGamma) – JodaTime and Java 8 Date and Time lead
See also the Devoxx UK website for more invited speakers.
As the call for papers will close on January 31st, I am sure some more big names will be added to the schedule.
Can Devoxx London create the same buzz? It is up to you to make sure this happens.
Prize Draw
Devoxx UK has offered us a free ticket. What can I do with one ticket – a prize draw perhaps! How to enter – it is very simple: On Twitter re-tweet my message @josetteg. On Facebook add me, share my post with all your friends and comment telling us why you should win the free pass. The winner will be announced on February 15th – to give you enough time to warn the boss that you need two days off.
The draw is for a free pass to the conference, the prize does not include travel, catering or accommodation expenses.
In 2010, ZeroMQ was still a little-known networking library. Since then it’s grown to define a new set of patterns for writing scalable software. Recent projects like Axon, JeroMQ, NetMQ, NullMQ, Crossroads, and Nanomsg are inspired by, or directly copied from ZeroMQ.
What makes ZeroMQ so popular and controversial? I’ve watched and been part of this story for a long time. My little software firm, iMatix, has always built distributed software, as far back as 1990. We know how difficult and expensive it used to be to get code talking to code. In 2004 we started designing AMQP for JPMorganChase, and building the OpenAMQ messaging server. In 2007 we felt that wasn’t going to fly, and began building a brand new open source stack focused on performance, simplicity, and decentralization.
ZeroMQ’s appeal and controversy comes from how cheap it makes this task. It’s difficult to believe the first time you see it, and many people don’t believe it. It can be offensive to people who’ve spent years learning technologies, or millions of dollars to buy them, to hear, “well, actually there’s this software library that does all of this, faster, better, and cheaper by ten to a hundred times”.
Scalability is about cost: if you can bring the cost of something down, you can do more, or make more of it. Making distributed software was expensive before ZeroMQ came along. It made connectivity cheap by solving a bunch of the hardest technical problems–such as sending messages asynchronously in the background using a dedicated I/O thread without locks, or distributed a message to specific subscribers–and wrapping the results up in a simple API that looked a bit like “classic” BSD sockets.
Smart engineers and businessmen are learning and using ZeroMQ to build products like Storm, dotCloud, Mongrel2, ZeroRPC, SaltStack. When you make a new software product, you don’t really need to know how you’re going to make money from it if you have grossly inefficient competitors to attack. When MySQL got initial funding, their story to investors was, “There’s a $100 billion market for databases. We intend to reduce this to $10 billion, and take twenty percent of that”. People who still aren’t using ZeroMQ are paying too much to build distributed or multithreaded applications.
So what are ZeroMQ’s strengths and weaknesses? Let me start with its main weakness: it doesn’t do security. Like TCP or any other networking protocol, this has to be layered on top. It’s still something we’re learning how to do properly. Its strengths are its patterns and transports, which you use like building blocks to connect threads, processes, or boxes. These patterns have become a sort of common language for distributed systems.
The other strengths are, I think, its community and wide support for languages and platforms. When you read the ZeroMQ book you’ll see that the examples are in C. But the same patterns work in over 40 languages. I also really like using ZeroMQ to do multithreading. That used to be the nastiest part of programming but when you switch from locks and mutexes to messages, it’s like switching from a cold shower to a nice warm one.
Messaging is a difficult problem to understand, let alone solve.
Connecting two pieces, OK. But connecting five thousand? It’s mind-boggling. Thus, many people who are getting into messaging will use a product like RabbitMQ (which was the second AMQP product after OpenAMQ). RabbitMQ makes big problems simpler by putting a broker in the middle and mandating, “This is the centre of everything”. It’s like saying, “The centre of the Internet is Facebook”. Simpler, if not totally accurate. RabbitMQ is a great piece of software but I’d never use it. It would be like using Subversion again when I’ve understood how Git works.
But the real reason why I recommend you download and try ZeroMQ, if you didn’t already, is the sheer pleasure of writing code using it. It takes a few days of coding and then you will experience a kind of enlightenment, and see new dimensions in your software. Not only can you make much more ambitious architectures, but you can make them easily, and the results are fast and solid.
For me, after working hard on OpenAMQ for years, and then having to kill the project as AMQP went off the rails, ZeroMQ saved me from professional burnout. It made coding fun again. It made me giggle as I saw 50-line C programs do stuff that used to be impossible with thousands of lines. I hope the book carries that enthusiasm forward and that you enjoy using and contributing to ZeroMQ as much as I have.
Pieter Hintjens started his first business making video games 30 years ago and has been building software products since then. Believing that, “the real physics of software is the physics of people”, he focuses on building communities through “Social Architecture”, writing, and building his businesses. He is the lead maintainer of ZeroMQ, and CEO of iMatix Corporation. Pieter speaks English, French, Dutch, and pieces of a dozen other languages. He plays with a West African drum group in Brussels, is a certified pistol instructor in Texas, and is currently learning Korean. He lives with his lovely family in Brussels, Belgium but travels extensively for work.
The lean enterprise middleware company – WSO2, showcases its revolutionary architecture, agile startup methodology, and open source culture at the third annual user conference, WSO2Con. With the event just two months away, the energy and excitement certainly is building up. We are happy to extend a 15% discount on the registration fee to valued friends of O’Reilly – members of our community. Simply use promo code WCPO15 at the time of registration.
WSO2Con 2013 will be held February 12-14, 2013 at the Park Plaza, Victoria in London. Guest speakers at WSO2Con 2013 represent some of the world’s innovators in the development and implementation of SOA, Web services, open source, big data, cloud, mobile and other software technologies. These experts will share their insights in presentations across a number of the conference’s keynote, technology and industry tracks.
Keynotes
- Democratizing Innovation in Computing, Eben Upton, Founder and Trustee, Raspberry Pi Foundation
- How Open Source Can Still Save The World, Brian Behlendorf, Senior Advisor for Science and Technology, World Economic Forum
- Network-embedded Application and Cloud Platforms: Emerging Foundations of Next-Generation Business Systems, Pankaj Srivastava, Vice President Engineering, Cisco Systems
Sessions
- API and Service Orientation @StubHub, Phiroze Dastoor, Senior Applications Architect, StubHub
- Strategic Insurance Integration Platform at Suva, Igor Berchtold, Senior IT Architect, Suva
- The Spanish Language API Platform, Elena Álvarez, Head of the Morphology and Syntax Department, Molino de Ideas
- Case Study: SOAS University of London Student Enrolment Portal on WSO2, Charles Gibbons, Technical Design Authority, JISC Advance
- How WSO2 Helped the Dutch Ministry of Infrastructure and the Environment Save Euro 500K and Deliver SOA Solutions Faster, Stephen Oostenbrink, Enterprise Architect, Yenlo BV/Dutch Ministry of Infrastructure and the Environment, and Thijs Volders, Senior Software Architect, Yenlo BV
- WSO2 as a Cryptographic Services Platform, Roger Carhuatocto, IT Consultant, Konosys
- SOA That Works, Alberto Lagna, CTO, Biznology
- Developing Innovative Telecommunication Products by Transforming Business and Operational Service Platforms, Neeraj Satija, Software Development Manager, Two Degrees Mobile Limited
- Banking on the WSO2 Platform to Build a SOA Strategy, Diogenes Justo, IT Manager, Banco Indusval & Partners
- Building Self-managing System Using WSO2 BAM and WSO2 CEP, Prajod Vettiyattil, Lead Architect, Wipro Technologies
- A Discussion on the Modern Enterprise IT Management, Richie Etwaru, Director, Group Technology, UBS
- High Performance Cloud Integration Platform, Dipanjan Sengupta, Principal Architect, Cognizant Technology Solutions
Complementing the guest presentations will be a series of sessions and in-depth tutorials delivered by leading technical experts at WSO2, who have contributed to many of today’s SOA, XML and open source standards. Details of the full agenda.
QCon London 2013 Highlights: Future of Mobile Dev, Technology in Finance Tracks; New Speakers Added
The 7th annual QCon London will feature over 100 speakers presenting more than 80 presentations across 15 tracks. Last year’s event was completely sold out with over 1,200 attendees: technical team leaders, architects, project managers, and senior practitioners just like yourself building and managing large scale applications for the enterprise.
We have arranged for you to receive a special discount. You will receive an additional £100 off using discount code “oreilly” when registering for the conference. The code can be used anytime until March 4th.
Keynoting this year are Design Patterns & Extreme Programming Pioneer, Ward Cunningham; MIT Professor & Renowned Computer Scientist Barbara Liskov; and Perl Boffin Damian Conway.
Newly confirmed speakers include:
- Martin Thompson, High-Performance Computing Specialist
- Andy Gross, Co-Creator of Riak, Chief Architect at Basho
- Mark Nottingham, Principal Technical Yahoo!
- Jesper Richter-Reichhelm, Head of Engineering at Wooga
- Rickard Oberg, Developer of JBoss, Webwork, Xdoclet; Creator Qi4J
Handheld Banking and the Future of Mobile Development
This year’s event will also take a deeper look at the evolution of technology in finance, including handheld banking, and the next generation of Mobile apps. Related sessions include:
- Advanced Sensors in Mobile Devices – Adum Blum, CEO of Rhomobile
- New Capabilities of HTML5 Browsers – Maximiliano Firtman, Adobe Community Professional
- A Look Inside The Financial Times Mobile App – Robert Shilston, Director, Labs Division, Financial Times
- Testing iOS Applications – Graham Lee, Security consultant and author of “Professional Cocoa Application Security”
- The Future of Mobile Banking – Michael Nuciforo, Global Mobile Thought Leader & Futurist
- Put a UI Developer in a Bank; See What Happens – Horia Dragomir, Metrics-driven game development
There is no other event in the UK with similar opportunities for learning, networking, and tracking innovation occurring in the enterprise software development community.
We hope you can join us for this year’s QCon London. Register before March 4th and save £100 with the discount code “oreilly“.
During this festive period, I hope you will have a little thought for our friends who are destroyed every day through deforestation, pesticides, greed and much more. Help ensure that our children/grand-children are able to admire those great endangered animals. Sorry little turkeys!
I might have said previously that Codebits is just about hacking, working day and night. Wrong! There are also great diners and and even greater entertainment. See the video where Jose Alves de Castro made me feel like a fool, in a nice way – I did not see the cards changing hands…
… and I was not the only one to be fooled. I am told to do these tricks, one needs about 300 hours of practice, be completely focused and maybe be a little eccentric.
PS. The video was taken by Carlos Morgado and not by me as wrongly stated previously. Both Jose and Carlos are very well-respected employees of SAPO, the organizer of Codebits.
London Perl Workshop
A cold, wet Saturday morning saw me once again making my way to Westminster University for the London Perl Workshop. Even though there were a lot of floods in the west counties, I got a train very quickly to Paddington and promptly got lost. Found my way and before I even got in the building was welcomed by Piers Cawley, parking his little car in front. I started to really feel at home. I won’t describe the setting up – you’ve seen one, you’ve seen them all. Suffice to say that I met some good, old friends –
- Mark Keating from Shadowcat Systems Ltd, organiser of the LPW and Perl marketing guru
- Dave Cross from Magnum Solutions, seasoned Perl tutor and author of two successful Perl books
- Michele Beltram, organizer of the Italian Perl workshop
- Laurent Boivin – the man with the money of the Mongueurs de Perl
- Liz and Wendy from the Netherlands – not only the most respected duo of the Perl community but always so generous. Liz and Wendy, if you are reading this post please be assured that my family and I will toast you on New Year’s Eve with the nice wine of Perl.
The list could go on for a long time but it is enough to say that I spent a great day with old friends. I also learnt that Piers is writing a little book for us.
Tech Mesh
I have been to Erlang meetings before but Tech Mesh is not just Erlang – it encompasses all functional programming languages. I have met a lot of new interesting people and a few O’Reilly and Pragmatic Bookshelf authors:
- Francesco Cesarini from Erlang Solutions, organizer and also co-author of Erlang Programming
- Joe Armstrong (a.k.a. the grumpy lovely old man), Programming Erlang
- Dean Wampler – Programming Scala, Functional Programming for Java Developers, Programming Hive and many more
- Phil Wadler – Java Generics and Collections
- Jim Webber – REST in Practice, and I understand is presently writing another book with Ian Robinson – more on that at a later date.
- Bruce Tate, author of this event’s bestselling title – Seven Languages in Seven Weeks.
I also met Pieter Hintjens whose upcoming book ZeroMQ should hopefully be ready for Fosdem.
One of the main attractions was the meeting of the grandpas of Erlang (with a combined 183 years of programming): Joe Armstrong, Mike Williams and Robert Virding.
Oops! almost forgot to mention the meeting of the 3 Haskell gurus’ keynote – John Hughes, Simon Peyton Jones and Philip Wadler – Haskell: Practical as well as Cool.
I looked at the time and realised the afternoon had flown by, my wife would be leaving work soon. I was just about to put the kettle on when I spotted she was still in work. How did I know? It was obvious from my WhereDial. I started to worry that we’d be eating late but the dial started moving and came to rest on “Travelling”. Not such a late one after all. I’d learnt the pattern a long while ago. It would tick round to “Station”, back to “Travelling” and then once it hit “Station” again it was time to put the kettle on as she’d be through the door shortly after.
This might sound like an odd story if you’ve never come across the WhereDial but a story similar to this has been playing out for me since I came up with the idea for the WhereDial. The original concept came about when I was working to promote my life mapping site MapMe.At and attended an Arduino hack day – Howduino – in Liverpool. I couldn’t think of anything to make and someone suggested I make a location clock “like in the Harry Potter books”. During the event I managed to get something basic working and eventually got a functioning clock working that would connect to MapMe.at, find out my location and update its hands to show where I was. I blogged about the clock and pictures of it have shown up at various conferences. People loved it and I was repeatedly asked if I would make another one to sell. That model was beset with problems though, the motor continuously broke and I cracked the glass on the face while building it so I realised this was never going to be something I could replicate and sell for a reasonable cost.
A year ago I came across a clock made from laser cut wood with a dial with numbers on that rotated continuously to show the current time. The ease of building and replicating this made me realise it was an ideal candidate for a reproducible and sellable version of the location clock, and so the WhereDial was born!
Over the past year I’ve spent much of my spare time working with laser cutters, designing PCBs for a custom Arduino shield and writing and re-writing Arduino software. While there’s been various bumps in the road and it’s taken much longer than I’d hoped I’ve made some real progress. A key point was when at the end of August I finally ordered the PCBS, they turned up 2 weeks later and worked perfectly!
So now we’re a year on in the project, where am I and most importantly will I be delivering WhereDials in time for Christmas? I’m happy to say the answer is – Yes! I added a basic shop onto the MapMe.At blog in the past few days and will start shipping orders early next week. UK orders should take just a few days to arrive but if you’re ordering from further afield, get your order in early if you want to have it by Christmas!
If you find the WhereDial interesting then keep an eye on its twitter account http://twitter.com/WhereDial or the MapMe.At blog for more information: http://blog.mapme.at/ – and if you want to order one, then head over to http://www.wheredial.com/.
John McKerrell and friends will be exhibiting at MakerFaire UK on 27-28 April 2013 at the Centre for Life, Newcastle.