8/15/13

Big Data, Hadoop and Big SQL, A Crash Course at the IOD 2013


Big Data and Hadoop are now long-lasting buzzwords in the data processing community. Yet, few database practitioners understand what these technologies are, how to use them productively and how to integrate them into a conventional data processing landscape. It’s no wonder, as nearly all resources on these topics target software developers and not data professionals.

IOD 2013: big data, hadoop, big sql, big insights
At this year’s IBM Information on Demand Conference, November 3-7 in Las Vegas, I will be giving a tutorial that is addressing this concern specifically: we will approach Big Data and Hadoop technologies from the perspective of data professionals. We will introduce the key elements of the Hadoop ecosystem, the IBM’s enhancements and highlight the impact of these technologies on the data systems and practices in the enterprise.

For this tutorial, we use IBM BigInsights Hadoop system and besides exploring the common Hadoop features we delve into some of its unique enhancements.

Here is the overview of what we are going to talk about:


  • What is Big Data? For sure you could not escape the Big Data buzzword, but do you know what Big Data really is? Is your data Big? How about Medium data? Could you/should you apply Hadoop and its tooling to it? There are benefits even if your data is not huge!
  • MapReduce algorithm. At the heart of Hadoop is MapReduce, the algorithm for processing large data sets with a parallel, distributed algorithm executing on a cluster. Learn about this algorithm that brings scalability and fault-tolerance to variety of applications.
  • Hadoop. Hadoop is the framework that implements the common parts of the MapReduce. It provides the environment in which to run user Big Data programs. It is fault tolerant, it scales, it is cost effective and it can enable thousands of computers to jointly process data in parallel.
  • Hive and Pig. While Java APIs for Hadoop allow for a lot of flexibility, they are at a fairly low level. For data professionals, the productive way of approaching the Hadoop is at a higher level: Hive allows for a subset of SQL to be run over the files stored in Hadoop’s Distributed File System (HDFS), while Pig is a data flow language. See the characteristics of both and its strengths and weaknesses.
  • HBase. The database for Hadoop. Complementing traditional Hadoop processing, which falls into a category of batch processing, HBase is a database that provides online / real-time performance. It lies on top of the other Hadoop infrastructure and it is a distributed columnar database.
  • Big SQL. Of course, the most productive approach for a data practitioner would be trusted SQL, but plain Hadoop does not have this feature. IBM’s Big SQL extension to Hadoop provides SQL users a familiar environment to become productive with Hadoop and even to use the JDBC APIs. You will learn how to use Big SQL and quickly become productive with Big Data applications.

How about the labs? In the tutorial we will show hands on how to start exploiting the benefits of Hadoop using the IBM BigInsights Hadoop distribution. We will use the QuickStart edition where you can begin exploring Hadoop in a virtual machine - just unpack and run.  You will get the instructions on how to get it after the tutorial and run the examples yourselves.

I am looking forward to seeing you at the tutorial at the Information on Demand Conference, November 7th 2013. The tutorial is part of the Big Data and Analytics Tutorial Series. Register now here.

128 comments:

  1. Hi Vladimir,

    What is your take on Big SQL vs. Impala? How would you characterize Big SQL in Matt Aslett's taxonomy? "7 Hadoop questions. Q5: SQL in Hadoop, SQL on Hadoop, or SQL and Hadoop?"
    Cheers,
    Jim Tommaney

    ReplyDelete
    Replies
    1. I have read your blog its very attractive and impressive. I like it your blog.

      Java crash Course in Chennai Java crash Course in Chennai | Core Java crash Course in Chennai

      java online crash course java online crash course | Java J2EE Online crash course | JavaEE Online crash course

      Delete
    2. Big data is a term that describes the large volume of data – both structured and unstructured – that inundates a business on a day-to-day basis. big data projects for students But it’s not the amount of data that’s important. Project Center in Chennai It’s what organizations do with the data that matters. Big data can be analyzed for insights that lead to better decisions and strategic business moves.

      Spring Framework has already made serious inroads as an integrated technology stack for building user-facing applications. Corporate TRaining Spring Framework the authors explore the idea of using Java in Big Data platforms.
      Specifically, Spring Framework provides various tasks are geared around preparing data for further analysis and visualization. Spring Training in Chennai


      The Angular Training covers a wide range of topics including Components, Angular Directives, Angular Services, Pipes, security fundamentals, Routing, and Angular programmability. The new Angular TRaining will lay the foundation you need to specialise in Single Page Application developer. Angular Training

      Delete
    3. I am confident you've got a great enthusiast following there.Small Business ERP Software

      Delete
  2. The expansion of internet and intelligence in business process lead the way to huge volume of data. It is important to maintain and process these data to be efficient in data handling. Hadoop Training in Chennai | Big Data Training in Chennai

    ReplyDelete
  3. Nice post. Big data is a term that portrays the substantial volume of information; both organized and unstructured that immerses a business on an everyday premise. To know more details please visit Big Data Training in Chennai | Primavera Training in Chennai |

    ReplyDelete
  4. Big data is a sophiaticated technology that helps to maintain the huge amount of data set.
    Regards,
    JAVA Training in Chennai|JAVA Course in Chennai|JAVA J2EE Training Institutes in Chennai

    ReplyDelete
  5. Great article. I learned lot of things. Thanks for sharing.

    php training in chennai

    ReplyDelete
  6. Great article. I learned lot of things. Thanks for sharing.
    qtp training in chennai

    ReplyDelete
  7. This comment has been removed by the author.

    ReplyDelete
  8. The best thing is that your blog really informative thanks for your great information!
    erp software in chennai | erp solutions in chennai | erp software development company in chennai

    ReplyDelete
  9. Big data can be used to improve training and understanding competitors, using sport sensors. It is also possible to predict winners in a match using big data analytics. Future performance of players could be predicted as well. Thus, players' value and salary is determined by data collected throughout the season.

    We provide best Primavera Training in Chennai with affordable Primavera course fees

    ReplyDelete
  10. Great information. I have got some important suggestions from it.
    Web design institute chennai

    ReplyDelete
  11. Thank you for the good write up. It in fact was a amusement account it.Look advanced to far added agreeable from you!
    Best Digital Marketing Academy

    ReplyDelete
  12. Really an amazing post..! By reading your blog post i gained more information.
    Bulk SMS Chennai
    Internet Marketing Company Chennai

    ReplyDelete
  13. Despite the fact that Hadoop is a full-fledged platform for developing any applications, it is most often used in the context of data storage and specifically SQL solutions. Actually, this is not surprising: large amounts of data almost always mean analytics, and analytics is much easier to do over tabular data. In addition, for SQL databases it is much easier to find tools and people than for NoSQL solutions. To know more visit Active Wizards Despite the popularity of SQL solutions for analytics based on Hadoop, sometimes you still have to deal with other problems for which NoSQL databases are better suited. In addition, both Hive and Impala work better with large data packets

    ReplyDelete
  14. Interesting post! This is really helpful for me. I like it! Thanks for sharing!

    Webseiten Gestaltung Lüdenscheid

    ReplyDelete
  15. I found a lot of interesting information here. A really good post
    office 2010 professional plus key deutsch

    ReplyDelete
  16. Really useful information about hadoop, i have to know information about hadoop online training institutes.

    ReplyDelete
  17. Thanks For Your valuable posting, it was very informative
    Internet Marketing Dienstleistungen

    ReplyDelete
  18. Nice post about MSBI, are you looking for best msbi online training.

    ReplyDelete
  19. Your website content nice nice and interesting to observe.
    jobbörse Neunkirchen

    ReplyDelete
  20. This is most informative and also this post most user friendly and super navigation to all posts... Thank you so much for giving this information to me.. 

    rpa online training |
    rpa course in bangalore |
    rpa training in bangalore |
    rpa training institute in bangalore

    ReplyDelete
  21. Great Article… I love to read your articles because your writing style is too good, its is very very helpful for all of us and I never get bored while reading your article because, they are becomes a more and more interesting from the starting lines until the end.
    Best Devops training in sholinganallur
    Devops training in velachery
    Devops training in annanagar
    Devops training in tambaram

    ReplyDelete
  22. I would like to thank you for the efforts you have made in writing this article. I am hoping the same best work from you in the future as well. In fact your creative writing abilities has inspired me to start my own BlogEngine blog now. Really the blogging is spreading its wings rapidly. Your write up is a fine example of it.

    python training Course in chennai | python training in Bangalore | Python training institute in bangalore

    ReplyDelete
  23. This comment has been removed by the author.

    ReplyDelete
  24. This is a terrific article, and that I would really like additional info if you have got any. I’m fascinated with this subject and your post has been one among the simplest I actually have read.
    angularjs Training in bangalore

    angularjs Training in bangalore

    angularjs Training in chennai

    automation anywhere online Training

    angularjs interview questions and answers

    ReplyDelete
  25. Does your blog have a contact page? I’m having problems locating it but, I’d like to shoot you an email. I’ve got some recommendations for your blog you might be interested in hearing.
    AWS Training in Chennai |Best Amazon Web Services Training in Chennai
    Best AWS Amazon Web Services Training in Chennai | AWS Training in Chennai cost
    No.1 AWS Training in Chennai | Amazon Web Services Training Institute in Chennai

    ReplyDelete
  26. I wondered upon your blog and wanted to say that I have really enjoyed reading your blog posts. Any way I’ll be subscribing to your feed and I hope you post again soon.
    Web Designing Course in chennai
    Web Designing training in chennai
    Hadoop Training in Chennai
    Python Training in Chennai
    Web designing Training in Porur
    Web designing Training in Adyar
    Web designing Training in Tnagar

    ReplyDelete
  27. Superb. I really enjoyed very much with this article here. Really it is an amazing article I had ever read. I hope it will help a lot for all. Thank you so much for this amazing posts and please keep update like this excellent article. thank you for sharing such a great blog with us.
    microsoft azure training in bangalore
    rpa training in bangalore
    rpa training in pune
    best rpa training in bangalore

    ReplyDelete
  28. Really very nice blog information for this one and more technical skills are improve,i like that kind of post.
    Best Devops training in sholinganallur
    Devops training in velachery
    Devops training in annanagar
    Devops training in tambaram

    ReplyDelete
  29. Your music is amazing. You have some very talented artists. I wish you the best of success. Domain Name Transfer

    ReplyDelete
  30. It’s a shame you don’t have a donate button! I’d certainly donate to this brilliant blog! I suppose for now I’ll settle for book-marking and adding your RSS feed to my Google account. I look forward to fresh updates and will talk about this blog with my Facebook group. Chat soon!
    python training Course in chennai
    python training in Bangalore
    Python training institute in bangalore

    ReplyDelete
  31. I encourage you to read this text it is fun described ... xender download for pc

    ReplyDelete
  32. I love visiting sites in my free time. I have visited many sites but did not find any site more efficient than yours. Thanks for the nudge! Fencing

    ReplyDelete
  33. thanks for your information really good and very nice web design company in velachery

    ReplyDelete
  34. Excellent article. Very interesting to read. I really love to read such a nice article. Thanks! keep rocking. Movavi Slideshow Maker 5.4 for Mac

    ReplyDelete
  35. This is such a great resource that you are providing and you give it away for free. I love seeing blog that understand the value of providing a quality resource for free. Newton MRT Station

    ReplyDelete
  36. Thank you very much for this useful article. I like it. Phoenix Heights Bukit Panjang

    ReplyDelete
  37. Wow, cool post. I’d like to write like this too – taking time and real hard work to make a great article… but I put things off too much and never seem to get started. Thanks though. Kampong Java Bid Newton MRT Station

    ReplyDelete
  38. Thank you for sharing such a nice post!

    Softgen Infotech is the Best SAP S4 HANA Training in Bangalore located in BTM Layout, Bangalore providing quality training with Realtime Trainers and 100% Job Assistance.

    ReplyDelete
  39. Hi...Nice Blog. You have shared useful information for beginners who want to start their carrier with Information and Technology. Big Data Hadoop is such a trending now a days. There are so many institute are available in your cities which will provide you crash courses for this recent technologies. You will be trained by industries experts and can gain a good knowledge related to your chosen field.

    ReplyDelete
  40. Thanks for sharing the post.. parents are worlds best person in each lives of individual..they need or must succeed to sustain needs of the family. offshore software development services

    ReplyDelete
  41. This is my first time i visit here and I found so many interesting stuff in your blog especially it's discussion, thank you. air quality monitor

    ReplyDelete
  42. Thanks for such a great post and the review, I am totally impressed! Keep stuff like this coming. remote team management

    ReplyDelete
  43. Thanks for uploads,
    https://softbuff.com/videoscribe-for-mac
    Videoscribe for mac free download We like it this page!

    ReplyDelete
  44. Break down creative barriers with CorelDRAW 2020 for Mac. Vector illustration, layout, photo editing, typography, and so much more. CorelDRAW Graphics Suite 2020 - FREE Download of Your 15-Day Trial! Design for print or Also available for Mac! Download Your Free CorelDRAW Trial.
    100% Working Setup ( Click Upper Link ). Download CorelDraw 2020 Mac Free is setup of standalone compressed file.
    Download a free, fully functional 30-day trial of any of our software products. No credit card CorelDRAW Graphics Suite 2020 for Mac.
    All links for Downloads:

    CorelDraw 2020 for Mac free download!
    CorelDraw 2020 mac download!
    link text
    CorelDRAW 2019 for Mac free download!
    CorelDraw 2020 for Mac
    !

    ReplyDelete
  45. This comment has been removed by the author.

    ReplyDelete
  46. This is such a great resource that you are providing and you give it away for free. I love seeing blog that understand the value of providing a quality resource for free. security guard license

    ReplyDelete
  47. I really appreciate the kind of topics you post here. Thanks for sharing us a great information that is actually helpful. Good day! Gift for woman

    ReplyDelete
  48. Thank you very much for sharing such a useful article. Will definitely saved and revisit your site handmade necklace

    ReplyDelete
  49. wow, great, I was wondering how to cure acne naturally. and found your site by google, learned a lot, now i’m a bit clear. I’ve bookmark your site and also add rss. keep us updated. Engine Cleaning

    ReplyDelete
  50. Pretty good post. I just stumbled upon your blog and wanted to say that I have really enjoyed reading your blog posts. Any way I'll be subscribing to your feed and I hope you post again soon. Big thanks for the useful info. Tableau Data Blending

    ReplyDelete
  51. I have read your article, it is very informative and helpful for me.I admire the valuable information you offer in your articles. Thanks for posting it.. Health

    ReplyDelete
  52. Your content is nothing short of brilliant in many ways. I think this is engaging and eye-opening material. Thank you so much for caring about your content and your readers. TrapandFitness

    ReplyDelete
  53. Took me time to understand all of the comments, but I seriously enjoyed the write-up. It proved being really helpful to me and Im positive to all of the commenters right here! Its constantly nice when you can not only be informed, but also entertained! I am certain you had enjoyable writing this write-up. Alcohol delivery London

    ReplyDelete
  54. It is perfect time to make some plans for the future and it is time to be happy. I've read this post and if I could I desire to suggest you some interesting things or suggestions. Perhaps you could write next articles referring to this article. I want to read more things about it! Alcohol delivery London

    ReplyDelete
  55. I don t have the time at the moment to fully read your site but I have bookmarked it and also add your RSS feeds. I will be back in a day or two. thanks for a great site. chubbies swim trunks

    ReplyDelete
  56. Great Article… I love to read your articles because your writing style is too good, its is very very helpful for all of us and I never get bored while reading your article because, they are becomes a more and more interesting from the starting lines until the end.
    oracle training in chennai

    oracle training in omr

    oracle dba training in chennai

    oracle dba training in omr

    ccna training in chennai

    ccna training in omr

    seo training in chennai

    seo training in omr

    ReplyDelete