Tuesday, August 23, 2016

Top 10 Big Data Facts That You Discuss At The Second Meeting

I was talking about Big Data to PhD students and early career researchers at the University of Melbourne on Friday, 19 August 2016. Here are the insights that I shared.

If you are confused after attending your first public lecture or vendor presentation on Big Data and not sure how much that relates to you. If you are still so puzzled that you have thousands of questions. You are not alone. I have been meeting many new Big Data followers having the same feeling. Based on that experience, here are my top 10 points that the new followers can make themselves aware of prior to that follow-up meeting. Ideally, these are the points that any independent experts would discuss with you in the second meeting.

1. There is nothing in the definition
Big Data is often described using its characteristics. Initially it used to be described in terms of Volume-the quantity, Variety-type of data and Velocity-speed of data generation. But as the time passed by, different vendors and practitioners started to add more characteristics to describe Big Data, such as, Variability-inconsistency of data, Complexity-difficulties to link available data and Veracity-quality of data. For the new followers it has been often bit confusing as they tried to relate their dataset to each of the above characteristics of Big Data.

However, the reality is, you do not have to have each and every one of the above characteristics in your data in order to manage it with a Big Data framework. One should simply be able to relate to some of the characteristics and be able to see the necessity, if any, for the organisation to use Big Data framework.

2. There are plenty of hypes and it may not apply to you
There are plenty of hypes around Big Data and merely having a Big Data framework in your organisation does not solve all of your data management and analysis problems. A very valid question to ask at the very initial stage, do we have an issue of the scale of Big Data in the current project? If the answer is a definite No, then move on. It is not the end of the world if you do not implement an organisational Big Data framework. However, it does not mean that you can ignore the importance of proper data management with the help of traditional frameworks.

3. Be realistic
One has to be realistic about what they can get out of their Big Data framework and process. For example, predictive analysis is something very popular among Big Data devotees. But the output of any predictive analysis depends on contexts, .e.g. accessibility of all the necessary data including external datasets or skill level of the analysts in the team. You may have to allow some time to finally get the best outcome of your Big Data investment. The lesson is to keep your expectation in check and to be patient.

4. You need to understand your business domain
To get the best out of your Big Data exercise you need to understand your business and organisational practices and vision well. Additionally you will also be required to make your best effort to understand your competitors and peers. As leader, at the bare minimum, you have to have a comprehensive list of your organization’s data repositories, sources and targets. Once you know what you already have, you can concentrate what else you may require to move your business forward inline with your organisation’s vision. It is very often the case that you will be pleasantly surprise to see insights from the data that you already have.

5. Creativity should and will always prevail
In order to leverage the huge potential of Big Data you need to be creative. You need to have creative minds in the team, which in turn will drive your analysis and future directions. You need to invest in people and need to retain the right people in the team who overtime are more likely to be able to produce critical insights from your data.

It is very popular to hire an external consultant for short period of time to find insights from your data while the existing employees are doing their routine work. This approach is not always productive. My observation is, whenever an existing employee, who possibly worked for the company for few years, is given the opportunity and allowed to work in finding insights from data, it has been faster and more productive.

6. You don’t have to tell good or bad stories, you have to tell stories out of your data
Often organisations themselves or through external consultants try to find insights from their data in form of either positive or negative stories. This approach mostly tells you stories that you intended to find. It is a bias approach to start with.

In particular, when an organization is not doing so well, they are so focused on what is going wrong that they do not see what is working for them and do not see any potential bright spot. The bottom line is, you simply need to tell stories out of your Big Data, they don’t have to be either positive or negative.

7. You don’t have to analyse the whole dataset at any given time
“I just have learned about all the external data sources on top of my own organisation’s datasets. How is it possible to store, analyse and find insights out of them all? Even though the Big Data presentation promised to deliver, will that outweigh the benefit for my organisation?”

These are some of the most common thoughts that the new followers generally envision right after an overwhelming Big Data presentation. To get you out of your misery, I can categorically ensure that you do not have to analyse each and every dataset that you have access to, every time you needed some insights. It is always been about identifying a possible subset of your data for a given insight that you have in mind and then to let the analysis of that small subset lead you to other datasets, if necessary.

8. Their commercial product is not the solution to your Big Data problem
If you were at a vendor presentation and were listening about their products, it is almost certain that you were told their products have solutions to all of your Big Data issues. That may not always be true. The solutions to your problems lie within. You have to identify the issues and have to detail a vision to overcome the problems, independent of any technology. Then and only then, you identify a tool that can help you achieve your goals. This approach will not only solve your Big Data problems with minimum efforts but will also deliver you the best out of your investment.

9. Big Data is not about checking your business health against Social Media sentiment
This is one of the top misconceptions floating around Big Data. It is mainly fuelled by the examples that speakers often use in their presentation while describing Big Data. Social media data should only be seen as a very tiny component of your Big Data framework. By all means, social media sentiment will provide you with useful insights but that should not be the benchmark and Big Data does not encourages you to run your business based on social media sentiment. In fact it is risky for your business decision making to depend solely on social media sentiment analysis as it is often can be bias.

10. Data management or storage itself is a huge task
Finding interesting insights from data has been the most exciting takeaway from Big Data lectures and rightly so. In the process, data storage and management were often overlooked and new Big Data devotees pay less attention to it. But, it is very important to realise that before finding any insights out of the available data, one has to have plans for data storage and management. The reason is simple, if you do not have any suitable placeholder for your data i.e. if you cannot store, access and secure your data, you will not be able to run any analysis on it.

Knowing the above facts will help you identify the right solution to your Big Data problem. And remember Big Data is not limited to your outgoing channels of your organization, such as, sales or marketing. It can also be useful to understand your internal resources, such as, employee satisfactions and needs. So Happy Exploring Big Data!

Sunday, April 24, 2016

Do we need Agile development?

I have written this article for the CIO Magazine Australia in 2014. The published CIO link can be found at http://www.cio.com.au/article/538077/guest_opinion_do_we_need_agile_development_/

Agile isn’t necessarily all it’s cracked up to be, says Dr Arif Jubaer

 As a keen observer of the IT industry, I come across numerous buzzwords and phrases to describe new and improved technology products, services, and implementation methods.

In the software development world, one of the most prominent buzzwords has been “Agile” or the Agile Software Development Methodology, which enables developers to constantly assess if a project is on track through the entire development cycle.

Words and phrases like interactive and incremental, accommodative, cost-effective, faster development lifecycle and reduced project failure rate – among others – are used to sell the Agile concept. It’s even been called the ‘smart’ methodology.

But quite frankly, Agile is just another methodology not a solution to all your software development challenges.

Let’s start with iterative and incremental development where a little more is added into design and development each time more requirements are understood.

If we look closely, this nature of the Agile model combines the phases of the Waterfall model, such as, analysis, design, development and testing within an ongoing loop. It does not offer or add any new phases or elements.

A good project team should always be aware that requirements will change as they design and develop. The team should be smart enough to accommodate the changes with minimum disruption within the existing software development phases irrespective of which methodology is followed.

Agile also allows users to respond to features and review the product for changes on an ongoing basis. This is certainly a good feature but development teams often struggle to think about the bigger picture while they are busy addressing these ongoing reviews. This is because they are focused on the current agile iteration goals to avoid frequent negative feedback from the development manager or users.

Also, over-involvement of users do not always bring simplicity – it can introduce complexity as users constantly change their minds. Thus, software development 101 teaches us that there has to be a cut off point for analysing requirements or adding new features.

Add a large scale development team to this scenario and the accommodative features of Agile diminishes as the complexity outweighs the benefit.

Is Agile more cost-effective?

The cost-effectiveness of agile is also often misinterpreted. Have you noticed that the testing team is always involved as per the methodology or that the Agile coach often consumes a share of your mainstream development budget?

You may also need to buy extra stationery to create and manage an Agile story wall, as well as software to manage the Agile process. Don’t forget, you also need to pay the project lead!

Is Agile faster?

Agile apparently builds faster and requires less or no documentation – far less than Waterfall.

Agile practitioners often disagree but when the methodology is put into practice, the development team struggle to keep pace and documentation suffers.

At the cost of lesser documentation the users may get quicker temporary solution but in the long run, they suffer more when they need to change the software functionality.
This also has bad impact on the development team who built the software at the first place because it becomes much harder for them to remember the existing functions of the software without the right documentation, or worse, if the development team has changed.

Does Agile reduce the project failure rate?

Reduced project failure rate is another highlighted characteristic of Agile. There is no guarantee that a project will not fail if you follow Agile. A project that failed using Waterfall won’t necessarily be successful with Agile.
That prompts me to point out few reasons why project may fail, such as the incorrect assumption that requirements are fully understood at any given time of the project, that changes will always be manageable, and integration will be smooth.
In relation to these wrong assumptions, Waterfall never forbids the development team to keep understanding the requirements as they develop. It doesn’t prohibit changes from being addressed in small chunks.


Agile more suited to extroverts

Agile is more suitable for extroverts who can promote their work well during the Agile standup. But to the best of my knowledge often most of members of a development team are introverts.

These introverts – who are often very creative – contribute equally like any other team members and with Agile, their contributions often go unnoticed to the demanding project lead.
Also, the less confident a project lead is, the more he or she likes Agile. I believe this is because each day, the entire team reports to him or her during Agile standup meetings, which gives the project lead a sense of satisfaction.
Ultimately, organisations should invest more in preparing leaders and creative teams and should not invest too much in another methodology that may or may not work for them.
They should find the best way for rational communications between team members and the business without any regular overhead.
It’s important to think about long-term impacts as well, instead of opting for a quick solution at the cost of documentation or any necessary elements, and don’t be influenced by competitors and the methodologies they are pursuing.
If Agile is the only best way forward for a given scenario, then go for it. But you do not need to fit into the Agile box. Creativity should take the upper-hand instead of the so-called ‘smart’ methodology. And the entire team should be vigilant at any given time to achieve short and long term application development goals.
Dr. Arif Jubaer is the founder of Daily Positive (D+). He also holds a PhD in IT from the University of Melbourne.

Saturday, April 23, 2016

Daily Positive (D+) Carnival Middle-East 2016

Another year, yet another colourful D+ Carnival!

The 2016 D+ Carnival was held throughout the month of March 2016. Every year, the carnival planned to promote and share focused positive news and stories from a specific location in the world. This year we celebrated the joys and delights of the Middle East.

The 2016 carnival also wanted to attract everyone’s attentions to two different D+ global campaigns #BringBackOurGirls and #StopBurningMyChild. Both the campaigns need continued attentions from the global citizens in order to save lives of children, women and men. D+ believes celebration is a strong form of demonstration of support for any burning cause.

Visit http://www.dailypositive.org/Daily_Positive_Carnival for more colourful pictures from the Carnival Middle-East 2016.
Join us again next year in the celebrations and continue to discover and share the positivity on D+.

Sunday, November 01, 2015

Switzerland Named World’s Most Positive Country For 2015

Switzerland is the World’s Most Positive Country for 2015, according to the annual global index published by Daily Positive (D+), a unique online media organization from Australia for positive news.  It is the first time Switzerland took the top spot in this annual positivity index.

The annual list of top ten most positive countries is compiled based on a positivity index comprised of a global expert panel rating, a D+ journalists rating, a global web poll and the number of positive news entries in 2015 on the D+ website.

The top 10 World’s Most Positive Countries for 2015 along with respective ranking points are as follows:
Country Name
Total Points
United States of America
New Zealand

Apart from this overall list of ten most positive countries, for the first time ever, D+ also introduces a list of regional positive countries from Africa, Asia, Europe, North America and South America. The regional winners were selected based on the same criteria used to select the overall winners but from a specific region. The winners earned their places with some amazing positive initiatives and actions in 2015.

Switzerland jumped to the top spot from last year’s fourth position with its consistent display of competitiveness in nurturing innovation and talent. Such sincere support helped projects like the solar-powered aircraft Solar Impulse 2 or the Large Hadron Collider to go to a new high this year. Switzerland also ensured the country’s productivity and prosperity by ranking higher in various prestigious indexes, such as, global democracy index, peace index, tourism-friendly index and education index. “Based on innovative environmental management practices, Switzerland has become one of the most sustainable nations in the world in the areas of climate change, biodiversity and habitat protection”, says one of our expert panel members.

Second placed Sweden was not far away in terms of innovation and also in all the above mentioned indexes. Perhaps that is why this year the Swedish researchers won prestigious awards, such as, the Princess of Asturias Award for Technical & Scientific Research and the Nobel Prize in Chemistry. Sweden’s Eurovision song contest win made its long list of achievements for 2015 more entertaining. Sweden maintained its leading position in media transparency.

China moved to third position in 2015, sliding down one spot from last year. One of the important diplomatic wins for China in 2015 was to be able to attract global interest to the Chinese-led development bank Asian Infrastructure Investment Bank (AIIB). China made significant progress with its domestic navigation system Beidou by launching new satellites and with the newly developed carrier rockets. Also, Youyou Tu of China won the Nobel Prize in Physiology or Medicine for her discoveries concerning a novel therapy against Malaria. This is the first time that any expert in Chinese traditional medicine has been awarded a Nobel.

After retaining the top position for two consecutive years, England moved down to the fourth position in 2015. Despite its slow productive run this year, English researchers unveiled a graphene light bulb with lower energy emissions, longer lifetime and lower manufacturing costs. English archeologists have discovered one of Britain’s largest medieval hospital cemeteries. English pride prevailed at the World Snooker Championship, Women’s European Hockey Tournament and Ashes Cricket championship.

Australia remained at the fifth position in 2015. Australian researchers have unveiled the world's first 3D-printed jet engine, the University of New South Wales and their robots defended their RoboCup World Football Championship title. Australia successfully launched one of the world’s most-advanced communication satellites Sky Muster into orbit, and they may have lost the Ashes to England but won their fifth One Day International Cricket World Cup at home.

Germany moved up one position to the sixth spot in 2015 compared to last year. Germany was well supported in our global poll for its role with the ongoing Syrian refugee crisis. German construction of wind turbines reached a record high, German researchers helped develop a prototype quantum radar that has the potential to detect objects which are invisible to conventional systems, and Germany has been highly tourist friendly.

The United States of America slid down to seventh position this year. Improved diplomatic relationships with Cuba and Iran were seen as the best foreign policy successes in 2015. Some of this year’s notable scientific achievements were, American doctors successfully transplanted, for the first time in the world, a scalp and skull while performing kidney and pancreas transplants. American researchers have discovered the world’s first warm-blooded fish – the opah. Researchers at Massachusetts Institute of Technology (MIT) have identified a brain circuit that encodes time and place. According to one of our expert panel members, “USA, despite not being able to combat its economic deficiency with full potential, continued to maintain perhaps the best academic system that allows them to keep up the great works.” It is also notable that the United States of America won the Women's Football World Cup for the third time.

New Zealand made it to the eighth position in the list. New Zealand ranked high in the democratic and peaceful country indexes in 2015. New Zealand announced the creation of a 620,000 km2 Ocean Sanctuary in the Kermadec region, which will be one of the world’s most significant fully protected ecosystems. Also, New Zealand reported a budget surplus for the first time since 2008, meeting a target set in 2011 following the Canterbury earthquakes and the international financial crisis.

Singapore is at the ninth position and made it to the list for the first time in our positivity index’s 5-year history. Singapore has been very competitive in terms of innovation and education. Singapore’s education systems, from school to the university, have been rated top-class in 2015. Also, Singapore‘s Botanic Gardens was recognized by UNESCO as the country’s first ever World Heritage Site.

Another newcomer to the list, Norway, took the final spot of our top ten positive countries. According to a report published in 2015, Norway had the highest proportions (35.5%) of women in corporate boardrooms compared with the rest of the world. Norwegian democracy was top ranked and served as example for the global communities. Finally, one of our expert panel members highlighted, “Norway topped the list of Save the Children’s 2015 Mothers’ Index, which ranks the well-being of mothers and children around the globe.”

D+ Most Positive Regional Countries 2015


Africa: Nigeria

Despite slow response to the rescue of the missing Chibok schoolgirls, Africa’s biggest economy Nigeria stood firm with new administrations and fresh approaches against terrorism in the country. Nigeria signed a bill that criminalizes female genital mutilation in the country. In case these were not impressive enough in order for Nigeria to take the regional title from Africa, it is worth mentioning, the World Health Organization announced that polio is no longer endemic in Nigeria, bringing the country and the African region closer than ever to being certified polio-free.

Asia: Bangladesh and India

Bangladesh and India were jointly named as the most positive countries from Asia. Both the neighbors simplified their border by exchanging more than 150 enclaves of land and settled the long running dispute peacefully. It was widely compared to the Fall of the Berlin Wall. Moreover, Bangladesh has become a lower-middle income country. Bangladesh has shown outstanding leadership on the frontline of climate change that bestowed Honorable Prime Minister of Bangladesh Sheikh Hasina with the prestigious Champions of the Earth Award by the United Nations. On the other hand, India inaugurated Cochin International airport, the first airport in the world that completely operates on solar power. India continued to march forward with their space missions and most recently successfully launched its first space Multi Wavelength Space Observatory ASTROSAT.

 Europe: Finland

Finland was our regional winner from Europe. Its reputable education system, vibrant democracy, peaceful and innovative approach to national and international issues earned them the honor this year. Women’s empowerment and professional development were also high on the agenda for Finnish prosperity.

North America: Cuba

Cuba was crowned as the regional winner from North America. Cuba restored full diplomatic relations with the United States and as a result, the United States removed many sanctions imposed on Cuba. Consequently, new travel and trade rules between the two countries came into effect, the diplomatic missions of each country became full embassies, and they have re-established direct telephone links for the first time in 15 years. Cuba is certainly back on the global stage with their own values and pride intact.

South America: Chile

Chile took the regional tittle from South America. Chile announced the creation of the largest marine reserve in the Americas in its Pacific waters. Chile won the Copa America football title for the first time in their history. Chilean scientists have been making the best use of the Atacama Large Millimeter/submillimeter Array (ALMA), an international setup of an astronomical interferometer of radio telescopes in northern Chile, with new and novel research outcomes. 
This year 4541 global citizens participated in our global poll with 61% men and 39% women. The global poll result suggests, Science and Innovation, Economy, Education, International Relation, World Peace, Environment and Sports were the most popular choice of positive accomplishments among both men and women participants while voting for their positive countries. Other choices were Freedom of speech, Tourism and Poverty Alleviation.

D+ continued to observe the trend on the percentage of people hopeful for positive future, primarily immediate future, based on the comments made during its global polls since 2011. It is observed that people’s positive outlook continued to decline since 2013 and apparently this year it was hugely affected by the ongoing Syrian refugee crisis. Perhaps that is why World Peace moved down as the fifth most popular choice of positive accomplishment in the 2015 global poll compared with being the first choice in 2014.

Apart from the winners, other countries have also shown tremendous resilience and achieved new heights. The small African island of Seychelles continued not only to attract tourists but also built solid diplomatic relationships with other countries. Japan’s win against South Africa in the Rugby world cup had to be the most spirited news in world sports in 2015. Tunisian National Dialogue Quartet for its decisive contribution to the building of a pluralistic democracy in Tunisia won the 2015 Nobel Peace Prize. Finally not to forget the heroic efforts that continued in Liberia, Guinea, Mali and Sierra Leone that almost eradicated Ebola virus.
“D+ World’s Most Positive Countries is in its fifth year and this year it has been celebrated with more enthusiasm and excitement indicating global citizens are more eager than ever before to have access to positive news. It is one of the biggest wake-up calls for world media to give positive news a chance to spread." says Dr. Arif Jubaer, Founder of D+.

D+ will present the above information and facts in detail at a presentation ceremony on 25 November 2015 at the University of Melbourne, Australia.
For more information on the World’s Most Positive Countries 2015 visit http://www.dailypositive.org/Most_Positive_Countries


Contact Information:

Contact Person: Dr. Arif Jubaer, Founder, Daily Positive (D+)

Mobile: +61-411215302