How to structure machine learning Problem

Hello world of machine..

One of the best article I saw for beginners,

https://machinelearningmastery.com/machine-learning-in-python-step-by-step/

Machine Learning Notes - 03

What is the process for a machine learning project,

simple and useful explanation,

"

5-Step Systematic Process

I liked to use a 5-step process:

1. Define the Problem
2. Prepare Data
3. Spot Check Algorithms
4. Improve Results
5. Present Results...............",

Read all

https://machinelearningmastery.com/process-for-working-through-machine-learning-problems/


Problem definition process..

Problem Definition Framework

I use a simple framework when defining a new problem to address with machine learning. The framework helps me to quickly understand the elements and motivation for the problem and whether machine learning is suitable or not.

The framework involves answering three questions to varying degrees of thoroughness:

• Step 1: What is the problem?
• Step 2: Why does the problem need to be solved?
• Step 3: How would I solve the problem?

Read all..

https://machinelearningmastery.com/how-to-define-your-machine-learning-problem/

Major Algorithm Families for machine learning...

https://machinelearningmastery.com/a-tour-of-machine-learning-algorithms/

Best Resources found on Machine Learning with Python

1. https://machinelearningmastery.com/machine-learning-in-python-step-by-step/

Machine Learning Notes 02

Very first thing you create with a machine learning program is a Data Model, which can use in business applications to do the predictions, forecasting and etc. 

There are four main steps in creating a Model ,

1. Load Data

2. Clean up data

3. Create the model

4. Train the model

5. Test the Model

6. Deploy the Model 

Step 1 - Load the data

To load the data, we use python library called "pandas" mostly

Example of loading data from a CSV file,

import pandas as pd

import matplotlib as plt

#matplotlib innline - this is just for Jupitor Notbooks

filename = 'datafile.csv'

columnames = ['preg', 'pres','skin']

data = pd.load_csv(filename, names = columnnames

Just to check the shape of the data, use,

print(data.shape)

and see the description of the data use

data.describe()

Can just the how data is been structured like this,

print(data.groupby('pres').size())

just the visualize the data,

use, Uni-variate and multi-variate plots

Uni-Variate

We start with some univariate plots, that is, plots of each individual variable.

Given that the input variables are numeric, we can create box and whisker plots of each.

code,

data.plot(kind='box', subplots = True, layout(2,2), sharex = False, sharey=False)

this plots a BOX plot for each numeric data field in the data set.

and view the histagram of the data by,

data.hist()

plt.show()

continuing....

Step 2 - Clean the data 

Before start processing data, clean up data is required in machine leaning. We do that in python with two main libraries,

1. numpy

2. pandas

Clean up of data can happen for different ways

1. Dropping unwanted columns from data frame

2. Changing the index of the data frame

3. Tiding up fields in the data

4. Combining str methods to NumPy to clean the columns

5. Cleaning the entire data set with applymap function

6. Renaming columns an Skipping rows

First import two main libraries,

import numpy as np

import pandas as pd

there is a function call "drop" comes with pandas to use for drop data columns from a data frame. 

first load the data from a csv file, repeat the code

filename = 'datafile.csv'

columnames = ['preg', 'pres','skin']

data = pd.load_csv(filename, names = columnnames

define the columns to drop from the data frame

drop_columns = ['preg', 'press']

then drop the columns like this,

data.drop(drop_columns, inplace = True, axis = 1)

Machine Learning Notes 01

Machine learning divides into two categories mainly,

1. Supervised Learning
2. Unsupervised Learning

Supervised Learning

Main scenarios of doing the Supervised learning,
1. Classification
2. Prediction
3. Sequence Prediction

Main methods of doing the Supervised Learning

1. Nearest Neighbors
2. Logistic Regression
3. Support Vector Machine
4. Random Forest
5. Naive Bayes
6. Neural Networks

Choosing the method to do the learning, change the mindset not to ask "How" and "Why",

but to ask "Does it work?"

Internet of things....

Internet of things... what are these things.. we know the internet... It's mostly like people connect to a one net..
this internet is not exactly same as the internet we know today... it is broader than that.

Things.. hmm things can be anything, it's not only humans, 

1. You

2. Me

3. My House

4. My Dog

5. My Car or any Car
6. Main light bulb in our house living room

7. Ultimately anything can sort of imagine.

Internet means they are connected to a one net where they can work together, collaborate, observe, control and etc from anywhere. Simply things can talk to other things connected in the net.

It's like devices in your home collaborate in sharing the electricity power. When washing machine is running, may be other devices will go on to a idle state if it is possible, to allocate more power for washing machine.

Internet of things is a concept that becoming real rapidly fast.

Here John Barrett talks about internet things.. interesting video to start learning the concept... 

Another interesting video about IoT,

Lakmal

Optical Character Recognition in .NET....

Interesting post about the optical character recognition in .NET

http://digit.lk/optical-character-recognitionocr-in-c/

Lakmal

Simple Made Easy

Very very useful talk by "Rich Hickey" about how we can make things simple in Software development, which will leads to make things easy.
Simple Made Easy....

Cheers

Lakmal

Analysis Paralysis

Over thinking or over defending things can prevent lots of winning moments, profits or new inventions than preventing failures. Detail analysis should come to play at the right time and in right phase of anything. This is not only about software developments, it is applicable to any industry.

Found a nice article about Analysis Paralysis, an idea which is applicable to any domain.

http://www.reliableplant.com/Read/18128/analysis-paralysis

Cheers

Lakmal

HTTP for Developers

HTTP, sometimes it is not quite noticeable that that HTTP is the paying a major role in allowing us to,

1. Learn the latest cricket news and Updates from CricInfo.com
2. Watch funny videos from YouTube.com
3. Search the we and find the stiff using google.com
4. and almost all whatever you do with the internet 

As Programmers, we got major responsibility of making opportunities to use Internet for people to achieve different different things in their life. Even though HTTP has been hidden from the Web Developers by the tools they use to develop web applications, still we need to know the basics, which will help to make things comfortable in,

1. Come up with new solutions
2. Troubleshooting the issues
3. And etc.

Here I found a series of Articles, which are explaining the HTTP for developers from odetocode.com. 

1. A Software Developer's Guide to HTTP Part I - Resources
2. A Software Developer's Guide to HTTP Part II - Messages
3. A Software Developer's Guide to HTTP Part III–Connections
4. A Software Developer's Guide to HTTP Part IV– Web Architecture
5. A Software Developer's Guide to HTTP Part V– State & Security

Truly useful..

Cheers 

Lakmal