Friday, December 11, 2015

Atlassian's success reminds what a crappy place Australia is for tech business.

Right now Australia is brimming with excitement at the IPO of Atlassian - Australia's greatest software company success.

Success has many fathers and no doubt the government will want somehow to be one of Atlassian's dads.

Whilst Australia should be immensely proud and admiring of Atlassian's achievements, we should in fact hang our heads in shame.

Australia is a crappy country for technology business.

Successive Liberal governments have been at best ignorant of technology and at worst openly hostile towards it.  The Howard's government's Senator Richardson had the reputation for being a luddite whose primary interest during the birth of the world wide web was to protect the interests of established media.

The Howard Government invested nothing of any significance in Australia's technology future.

Australia had a brief foray into being a world leader in technology when Kevin Rudd announced the National Broadband Network which would roll fibre optic cable to the vast majority of Australian homes.  Steve Wozniak liked it so much that it was reported that he was applying for citizenship on the strength of it.

Labor soon lost government to the Liberals led by Tony Abbott who ripped the NBN plan to shreds and with current Prime Minister Malcolm Turnbull instead gave us a half assed patchwork excuse for a broadband network, cobbled together from whatever bits of cable TV coax and telephone copper network were already lying around. 

In an age where the state of the art broadband to the home in the U.S.A. is 10 gigabits, Australia is currently building (it claims) a National Broadband Network at great expense that promises a blistering 12 to 100 megabits per second at the top end. And there's no sign of the NBN in most parts of Australia anyway because it's clear the government was really only keeping the project alive under duress.

The Abbott Government was deeply ignorant of technology.  Tony Abbott smirking his catchphrase "I'm no Bill Gates, but..." whenever any question of technology came up. No doubt about that.

Speaking of our fibre-to-the-home smashing current Prime Minister Malcolm Turnbull, he has just released the latest government innovation initiative and of course it doesn't do what is needed to genuinely foster innovation.  Instead, the government remains focused on giving money to risk averse venture capitalists and pointless incubators that are little more than desks and chairs. 

In 2015 innovation is as likely to come from one, two or three people sitting in the lounge room coding away.  This latest innovation strategy does nothing to support the thousands of programmers who just need some help getting the bills paid whilst they sit in the lounge room and code for 18 months to get that first version up and running.  

Instead it's the time honored formula of driving people to spend their time not coding, but writing business plans for the government funded venture capitalists to reject until the business is successful anyway (sorry "has traction").

I don't know alot about Atlassian but it looks to me like they did it with no help from the Australian Government nor from the venture capitalists who purport to support innovative businesses but really are focused on getting their next fund cashed up by the government. Atlassian's enormous success serves to illustrate that Australia has come up with only one Atlassian-sized success story. 

That's what we have to be ashamed of - we have created only one Atlassian.  Where are the rest?

Australia is a country so hostile to technology and so poorly supported by Government and bumbling venture capitalists that I can recommend strongly that if you are a young entrepreneur and have the means to leave, then do so as fast as you can.  

Get out of Australia and get to Silicon Valley.  Atlassian is an outlier - the exception.  Don't take Atlassian as a sign that you can do it from Sydney or Melbourne - you're not Steve Jobs, you're not Mark Zuckerberg and you're probably not the next Atlassian.

You'll have a much better chance if you get out to where things really happen. Atlassian has succeeded despite being an Australian company, not because it is an Australian company. 

Being an Australian business confers no particular advantage in the tech world. Get thee to Silicon Valley young person.

Wednesday, September 2, 2015

What "I don't have time" really means.

When someone explains why they didn't do something, one of the most common reasons is "I don't have time."

And it's a good reason.  Everyone understands.  Everyone is busy.  No-one has time.

Didn't get that work done?  Didn't have time.

Can't manage to see friends?  No time.

Never manage to see the grandkids? So hard to find the time.

Can't get to the gym?  No time for that either.

No one holds you to account when the reason is "I don't have time."  YOU don't hold you to account when you say to yourself "Why aren't I doing that important thing? Well I don't have time of course." Everyone understands.

But the truth is that "I don't have time." is just an EXCUSE.  It's a get-out-of-jail-free card. It's a socially acceptable way of concealing the less palatable truth of why something didn't happen.

When someone says "It's hard to find the time to do that", what they are REALLY saying is "That's not high enough priority to me.  Everything else in my life is a higher priority."

So next time you hear someone say "I don't have time", convert those words to "All the other things in my life are higher priority than that activity and that's why I didn't do it.".

There's the truth - everyone has time, it's just a question of whether they are willing to make it a high enough priority to actually do.

And next time you find yourself saying "I just don't have time for that.", trying saying instead "Everything else is a higher priority, sorry.".  Even if you only say it to yourself you'll get closer to the truth about why you do and don't do things.

Wednesday, August 5, 2015

What language has decorators, generators, comprehensions, modules, but isn't Python?

Question: What language has decorators, generators, comprehensions, modules, but isn't Python?

Answer: JavaScript of course! Specifically - the latest version of JavaScript - ECMAScript2015.

I've listened to a number of Python folks speaking recently and one of their most common refrains is "I hate JavaScript".  Python folks tend to be happy avoiding the browser end of development. And until recently, I was with them. I found JavaScript to be inconsistent, confusing, underpowered and frustrating.

But the latest version of JavaScript - ECMAScript2015 - is a game changer.  If you're a Python programmer you'll find that JavaScript and Python are starting to have alot more in common.

I've been programming recently with React.js in ECMAScript2015 and instead of burning in the ninth level of JavaScript hell, I've found that it's consistent, powerful and makes sense.  Sure there's a few concepts that are new but there's also a sense that JavaScript is becoming more Python-like.  In some ways it feels a bit like programming in Python.  I recommend it.

And yes you CAN use it today - it works in all modern browsers because things have changed - these days, ECMAScript2015 is transpiled/compiled into JavaScript, so no more fidgety worries about which browser supports what - write it, compile it, run it.

If you can live with all those damn annoying and pointless brackets, and a good IDE like PyCharm or Visual Studio will go a long way to sorting that out.

If you're adventurous enough to go further, this might help you get started:  

Tuesday, August 4, 2015

Microsoft Xbox Chatpad versus the obvious. Chatpad wins.

Microsoft has released the Chatpad for Xbox One.  A weird and clunky looking device.

It solves the obvious problem that it's hard to enter text on a console.

But the real obvious solution is to use a phone or tablet as a keyboard, not to come up with some weird boondoggle.

I am aware that there is Microsoft SmartGlass - an app for touchscreen devices that is an accompaniment to the Xbox.  Microsoft SmartGlass misses the point.  Instead of being feature-packed, Xbox Glass should have been nothing more that a keyboard and swipe interface for the Xbox One/360. Microsoft SmartGlass should have activated based on location proximity to your Xbox. Microsoft SmartGlass needed to be tightly focused in its functionality instead of being feature rich and diluted in its purpose.  It should have been a keyboard and swipe controller only.

But somehow at Microsoft, various committees thought about it and managed to come up with the very best solution - a clunky bit of hardware that you have to buy, instead of just using the device that everyone has in their pocket.

That's the Microsoft Way.

Thursday, July 16, 2015

Bare metal JavaScript is dead.

Bare metal JavaScript is straight JavaScript code that runs against the browser.  It's the way web browsers have been programmed virtually since the beginning. You write some code, the browser runs exactly what you wrote.

And bare metal JavaScript is dead.

You might not have heard the news yet.  This death is not sad news, it's great news.

You might still be working on a daily basis with the living zombie that is bare metal JavaScript, but believe me it is dead.

Bare metal JavaScript is now the equivalent of bare metal assembly language, and it has followed the same path to become just a target language for compilers.

In 2015 you no longer write bare metal JavaScript.  You write code in one of a selection of JavaScript successors, descendants and cousins, and that code is then transpiled to bare metal JavaScript which is what the browser runs.

If you haven't yet moved away from bare metal JavaScript to a transpiled language such as ES7, then its time you did.

And yes I know that ES7 is in fact the very latest version of the JavaScript language but so different and so much more improved and refined that it is almost a new language.

Tell your boss that you no longer wish to work with a zombie programming language and that you wish to move to ES7 or one of the ever-growing list of languages that can be transpiled to bare metal JavaScript. 

Monday, July 6, 2015

Why being on the JavaScript cutting edge matters.

The JavaScript world moves extremely fast.  Almost week to week a shiny new framework or library or build tool comes out.

Some people seem to feel that always moving to that latest shiny new JavaScript thing is self indulgent, trendy, hipster behaviour.

In fact it makes good sense to always be finding and staying at the JavaScript cutting edge. WHY it makes sense is because JavaScript is (or was) crappy, inconsistent, hard to use and quirky and because browsers are (or were) crappy, inconsistent, hard to use and quirky.

The JavaScript cutting edge in general tends to be finding more powerful and more simple ways of doing things.  If you want to make life as easy for yourself as possible, you need to live at the cutting edge.  

Living on the JavaScript cutting edge is not a matter of being cool or distracted or trendy - for me its a matter of necessity to avoid living in the burning hellfire of bare metal JavaScript programming, in which there is nothing between you and the browser except raw JavaScript. The truly lazy programmer lives at the cutting edge of JavaScript because that's where things are easiest - until a better idea comes along next week.

Sunday, July 5, 2015

History compression via bright, full colour technology

I was born in the late 1960's.  That's 20 years plus a a few years since World War 2 ended and the death of HitlerThat 20 year period seems like an incredibly long time to me.

Nowadays, 20 years doesn't seem so long. 20 years plus a few years was the early 1990's.  That's not long ago. That's when Jurassic Park and Terminator 2 came out. Pearl Jam and Nirvana were creating the grunge music thing. Heck it's in the ballpark of almost 40 years ago since Star Wars came out - I remember queuing all the way around a city block to get tickets. Now sure that seems a long way back, but not the distant past.  

I feel like history is being compressed by bright, full colour technology like film, colour TV and computers. Anything that happened before bright, full colour technology, heck that was LONG ago. Anything from the era or bright, full colour technology feels closer in time.

Tuesday, June 30, 2015

The biggest problem with react.js – knowing what is fresh.

Currently coming up to speed with react.js

Coming up to speed involves reading vast numbers of comment threads, StackOverflow question threads, the official documentation and blog posts from any number of authors.

react.js and its ecosystem are rapidly evolving.  Thinking changes, API's change, related projects change, approaches to solving problems change, build systems changes, everything changes extremely rapidly.  For goodness sake the JavaScript language changes.

The rapid pace of change is a good thing because the outcomes of change are usually better functionality or greater simplification.  

So you want to be working to the latest version of everything if you like to write simpler and more functional code.

BUT there is a HUGE problem. How the heck to know if what you are reading is outdated thinking or latest thinking? There is no way to know how near or far a given blog post or problem thread is from the cutting edge.

It's a huge problem because doing something "the old way" can send you down the wrong path for hours or days or worse. You almost always want to be doing things "the new way".

Any given page that you read might give you that answer to your current question or problem. But there's no easy way to know if a better way hasn't already been worked out by someone else.  It's a real problem.  I have no idea how to solve that problem. Do you?

Friday, June 26, 2015

The downside of being a full stack developer - lost productivity.

Full stack! Full stack! Full stack! Full stack! Full stack! Full stack! 

Employers love the idea that a developer knows everything.  It's almost an expectation.  

I've committed myself to being able to develop every aspect of my software which means I need something between effective practical to deep expertise in Python, JavaScript, react.js, HTML, CSS and a whole bunch of sub-technologies like SQL and the various libraries used to get any useful work done in any of these languages.  Oh yes also I need extremely deep knowledge of Linux and Amazon Web Services.

The price however is devastating in terms of time and productivity because these days getting anything done in any language requires not just that you know the language but that you know the entire ecosystem of tools.  Want to write a browser front end in reactjs?  Well you're going to need to have come to grips with npm, gulp, babel, browserify, react-router and ES2015. Want to write a Python back end?  Well you'd better have come to grips with virtual envs, uWSGI, some sort of web server like Flask or Falcon or Bottle or something as well as learned how to drive SQLAlchemy.  Want to deploy your system?  You'll need to know so many subtechnologies of Linux that's it hard to know where to start naming them.

I need to know so much stuff that by the time I have finished learning a new ecosystem I am worried that the one I was using two months ago has fallen out of my head and needs to be re-learned.

The upside of all this is great - I can build something without anyone's help.

The downside is the devastating cost of trying to grasp the mental model of every one of these development ecosystems and make them work, all the while fighting the tidal wave of errors and broken things that experts have become so good at navigating that they no longer notice any more.

I spend hours and hours and days trying to make things work and trying to fight my way to being able to write productive code instead of just making the damn technology work in the first place.

Presumably non-Full Stack developers are incredibly productive because they know their tools and technologies and can get on with the job of writing productive code, instead of spending all their time learning the latest wind of change to blow through the JavaScript world.  I envy them.

I seriously wonder if other full stack developers find the whole thing super-easy and breeze through it all learning everything in a flash and making everything work the first time.  Holy hoot I hope I'm not the only one who finds it hard to put all the pieces together.

Monday, June 22, 2015

Windows 10, the cluelessness continues.

In my previous post I griped about how Microsoft throws responsibility to you to work out how the heck to get their ISO onto a bootable USB stick - good luck with that one grandma.

The Windows 10 bootable USB stick that I tried to make on my Mac didn't work and I didn't figure out why in the end.  I eventually managed to get a bootable USB stick made thanks to extensive Google searching and thanks to a utility call "Rufus" which I ran from an old Windows 7 machine that I resurrected for the task.  Microsoft should just buy that company with its pocket fluff and stick the utility right there on the Windows 10 download page.  

So I booted up my laptop and was pleased to see the Windows logo as the installer loaded from USB. All went fine until I tried to get past the formatting of the hard disk.  It didn't work.  Windows gave me some vague and unhelpful error.  OK, so is this a problem with Windows 10 or is there something wrong with the new hard disk that I just installed into this machine? To find out I downloaded the latest Ubuntu Desktop installer and put it on a USB stick.  Ubuntu Desktop booted up and installed cleanly and had no problem with the hard disk. 

As an aside, I have to say at this point that I was a little taken aback. I have never had a good experience installing desktop Linux.  I've given it a go every 12 months or so for a few years and it ALWAYS feels clunky and unix-ish.  Usually, any flavor of desktop Linux throws reams of complex, frightening and technical sounding text messages at you, switches jarringly out of graphics mode into text mode and back again, fails to recognise your wireless network card, forces you through the most arcane possible hard disk formatting process.  None of that happened this time. Ubuntu Desktop installed easy, quickly, cleanly, the wireless network came up, it did not spew complex technical messages at me and managed to stay in graphics mode.  Within minutes I was browsing the web under Ubuntu Desktop. When I rebooted the machine it started so fast I was awestruck.  Hmmm.

Back to Windows 10 attempted install.....

OK so Ubuntu installs fine so theres nothing wrong with the hard disk.  I go back into the installer and try every possible option and variant on getting the disk to format.  I search Google for the error message, others on the net have had the same error installing Windows 8! I try some of the suggested fixes.  Nothing works.  Further Googling finds a thread somewhere that suggests that the problem is because it is a USB3 stick.  OK well I am using a USB3 stick which I thought was  a good idea because they are fast.  Alright so I go back and make a new Windows 10 installer on an old USB stick that I have. Boom!  Problem solved, Windows 10 formats the hard disk and installs.

Why should Windows 10 not install if the installer is on a USB3 stick?  I don't know and I really don't care.  Certainly grandma would never get past that problem in her quest to install Windows 10.  It's concerning though that it appears to be a problem that was seen in Windows 8 so it's an old bug.

I'm not a fan of Desktop Linux but I'm left feeling rather unsettled that the Ubuntu Desktop install worked a whole lot smoother and better than Windows 10 install. What the heck does that mean?  Windows used to be the go-to OS for PC's because it actually managed to make all that diverse PC hardware work - is that not true any more?

When eventually I got into Windows 10 I'm sorry to say it felt kinda clunky.  The thing that bothers me most about Windows these days is that it feels like its designed by a committee, like it has problems that are too big to fix, and it feels like the outcome of deep corporate politics rather than the implementation of a clear vision of what an operating system should be.  I'll use it, but only when I need to.

At the end of my Windows 10 installation adventure I was left wondering, if you sat 1,000 ordinary people down in front of 1,000 average computers from the past five years, how many people would succeed installing Windows 10 without expert help? I suspect fewer than Satya Nadella would like.

Saturday, June 20, 2015

Windows 10, the cluelessness shows at step one.

Cluelessness, what a great word, if it's a word at all.

I left the Windows world many years ago fed up with its general cluelessness.

I'm not your garden variety anti-Microsoft neckbeard.  For most of my professional career I was a die-hard Microsoft fan.  In fact I'm not anti Microsoft at all. I'd really like it if they got back in the game.  But many years ago the myriad problems got the better of me and I left Windows for OSX where things just worked.

Anyway having heard so many great things about Windows 10 I decided to give Windows a fresh go.  I thought maybe Microsoft has wised up and is doing things that make sense.  So I bought a new laptop with plenty of RAM and disk ready for my return to Windows.

Yesterday I started the install process by going to get the install files to copy onto my USB stick.

And immediately, before taking one single step back into the Windows world, before Windows 10 could run one instruction on my machine, I was smacked in the face with the old Microsoft cluelessness and I despaired.  Here's what Microsoft says on the Windows 10 download page:

That's right.  If you want to install Windows 10, before you do absolutely anything, before they let you in the door, before you can run one line of Windows 10 code, you must first carry out the deeply arcane, difficult and problematic process of finding some way of copying the Windows 10 install files onto a USB stick.  Microsoft kindly reminds me that "DVD burning software often includes this capability".   Thanks. When Microsoft says "third-party program" what they mean is "some other bit of software that someone else may or may not have written, we don't really care".

Eventually I found some instructions on the web from how to do this on my Mac.  Here it is, you'll need to open a Unix terminal and type this command on your Apple Mac to commence the process of installing Windows 10:

dd if=en_windows_10_pro_insider_preview_10074_x64_dvd_6651360.iso of=/dev/disk4 bs=1m 

There's one for your grandmother to type in to upgrade Windows. Needless to say, the USB didn't boot and I don't know why and I really am not interested in why.

In the fantasy world in which Microsoft had a clue, they would have absolutely made damn sure that it was incredibly easy to get going.  They would have provided incredibly easy and reliable ways to create that USB stick from any version of Mac, Windows, Linux, even from versions ten years old, cause that's how long its been since some people have upgraded Windows.  It would have taken a matter of minutes, with clear and simple instructions and no need to leave the page at all.

Instead I have spent hours trying to do create an installable USB from Microsoft's install ISO, something that I have zero interest in and zero time for.

Undoubtedly I have returned to the world of Microsoft and Windows. I now expect the old cluelessness to pervade as ever it did before.

Wednesday, June 17, 2015

Levelling up as a software developer by doing nothing

I've noticed a strange thing. 

If I stop programming with a given language for a while - say a period of weeks or months - when I come back to it I often feel like I have "levelled up". 

I feel like understanding has fallen into place and fundamental concepts and learning have become embedded in my mind.  This puts me in a mental state whereby I am able to learn a whole bunch of new concepts that I would not have grasped earlier.

Sunday, June 7, 2015

Why how and not why?

Software development in 2015 requires that you rapidly learn vast numbers of technologies enough to sew them all together.

The most frustrating part of trying to learn all this stuff is that most documentation is focused on how to do something but not why you are doing it.  Rarely is context given of why what is being explained needs to be done, only how it is done.

Very frustrating.  If you are writing software documentation please start every page, every paragraph, every example, every solution, with why before how.

Thursday, May 28, 2015

Governments around the world should build home broadband networks at $16,000 per mile

The U.S. government incentivised the creation of the transcontinental railroad by paying the railroad builders $16,000 per mile of finished track.

That's the right way to build home broadband networks too.

For example when an ISP proves that it has installed a broadband connection of a certain speed into a home, then the ISP should be paid $16,000 (substitute this for the appropriate incentive amount).

You get the outcomes you reward.  If governments want fast home broadband then that's what they should pay for.

Monday, May 25, 2015

Why do G-rated movies need to frighten the heck out of children?

My little boy has recently become old enough to enjoy going to the movies. What could be better than taking your kids to the movies on a Saturday afternoon?

But I've found that many, if not most, supposedly G-rated movies almost always contain frankly absolutely terrifying scenes that are often quite extended.  My boy jumps off his seat and hides and doesn't watch.

Many of these scenes are very, very scary. EVERY Pixar G-rated movie includes scenes that are terrifying to children.

But why? Why does Hollywood feel that every movie, even for tiny children, has to include deeply frightening scenes? I don't understand. It seems like the accepted wisdom is that "to frighten is to entertain".  

It's almost like Hollywood feels that movies just won't sell unless it can scare children to the bottom of their little souls.

Thursday, May 21, 2015

Its hard to learn the easy way to develop software.

The irony of modern software development is that it is hard to learn how to do things the easy way. What I mean by that is that the easiest and most powerful ways of doing development often require a significant base level of learning and knowledge to understand. You can take a faster route to getting something up and running but the price is that it will be harder to work with. Someone has thought of an easier way, but first you must understand it.

All of that means that your choices are to get something quick and easy up and running but not really have any valuable long term understanding of the best way to do things. Or you can go through a painful period of "learning the easy way".

I recently did this and invested significant effort in learning react.js for the browser, and Python Falcon server for the back end, with Python Sqlalchemy for database access.

The impedance mismatch between Linux and its software.

I do alot of compiling, installing and generally trying to make things work in Linux.

Generally things don't work.  

Installations almost always fail and then you have to start digging around to work out why.

The most common reason is that software expects certain files to be in certain locations but the expected files are not there.

After many, many failed installations I think its reasonable to say that there is a deep systemic problem with the way Linux software works. There is an impedance mismatch between the operating system and the software that runs on it.

This problem of "file X is not found where expected" is at the core of many problems with making software install and run.  Apparently software just doesn't fit very well within the operating system that houses it.

There must be a better way - for the next great leap in computing somehow we need to be able to make software that knows what it needs and always finds it.

Giant computer companies in race against governments to protect and encrypt.

Governments around the world are addicted to intercepting and spying on everyone's computers. They feel it is their right.

Before too long, governments will be requiring the computer giants to build back doors into their systems to allow the government in.

But this is not in the interests of the computer giants.  Inherently insecure and inherently eavesdropped computers and networks are of much less value to end users.

So the computer giants must hurry up and lock things down in a permanent and irreversible manner before the governments really get their legislative act together.  It's a race.

Wednesday, May 20, 2015

For Linux, your core skill is navigating broken stuff, which is everything.

In the Linux world, as others before have said, "everything is broken".

If you are a developer or sysadmin doing anything of any significance then your ability to reach your goal is defined by how quickly and easily you can recognise and find workarounds for broken stuff.

Actually, not everything is broken, but so much is broken that it feel like everything. It's the broken stuff that will take your time. The stuff that isn't broken will be easy and will happen quickly.

So the faster you can jump through Linux's never ending minefield, hot coals, machine gun nests and shards of glass, the faster you can get productive work done.

Wanna be productive?  Get good at seeing and sidestepping and fixing broken stuff.

Sunday, May 10, 2015

An incredibly brief introduction to JavaScript in 2015 for experienced developers.

Over the years I've done bits and pieces of web and JavaScript development.  For one of my current projects however I need to get some serious stuff done, so I knew I had to come up to speed on the state of the art in browser development. I chose to target react.js as it seems to me to promise the simplest mental model which is what I need because I am time poor.

And things have changed ALOT in JavaScript development since the old days.  It took lots of searching and reading to make sense of the current landscape so I will now save you days of effort and summarise what you need to know:

1: Programming a web browser directly with JavaScript is dead. In the same way that few people now programs CPUs directly with assembly language. When you are writing JavaScript it now gets compiled (actually transpiled) to JavaScript.  Forget writing code directly for execution in a browser, those days are gone, move on.

2: There are a number of JavaScript variant languages for you to choose to program in.  The mainstream choice is now ECMAScript2015, also known as ES6.  
ECMAScript2015 /ES6 turns JavaScript programming into something sane, for lots of reasons but primarily because it implements a module/import system. 

3: Your development workflow involves writing ES6 code then shoving it through a series of very useful tools that carry out transformations of your code. The main transformation is the transpiling (converting your ES6 code to plain old JavaScript e.g tools like 'babel'). Other transformations include minifying (compressing/reducing the size of the output, e.g. 'uglify'), uglifying or mangling (making your code to make it harder for others to make sense of - 'uglify' does this too), bundling (combining multiple JavaScript files into a single file to reduce load times e.g. tools like browserify), translation (if you are using reactjs then you can write JSX which is like including XML style HTML in your JavaScript, this also needs to be translated as part of the transpile process).  There's probably plenty of others but these are the main ones I have encountered.

4: Your development workflow will use some sort of tool to tie together all the transpiling and transformation  tools.  If you know what "make" is then you understand the concept.  If not, well it's just a utility of some form that has a few different configurations of the tools mentioned above, each configuration useful for pushing your code through the various utilities until a final JavaScript file pops out of the end. "gulp" is the tool that people seem to be using alot at the moment for this purpose.  

5: You will need to learn how to drive the Google Chrome debugger or spend a very long time in debugging hell.  Go to YouTube, search for "JavaScript Debugging Chrome" and spend a few hours watching.  Time well spent.  The key thing you need to know is that JavaScript "source maps" will allow you to debug your code in the browser even after shoving it through the various transformation tools.

6: You will want to make use of various JavaScript libraries.  This is done through the npm package management system.  Invest some time in learning the basics of this. The key thing you need to know is that you say "npm install <packagename>" which installs packages under a "node_modules" directory in your current directory, which is where you compile from. In your code you will use "import" and "require" to load in the packages.   

Hopefully that lightning introductions has helped you find your feet.

I'm learning all this.  Have I got it wrong, please let me know.

Thursday, April 30, 2015

Great software developers began non-technical & bulldozed obstacles to learning.

The coming micro service integration trend.

Currently there is a general trend towards Internet applications being very focused, minimalistic, sometimes even single-purpose. Examples include StormPath for user management, MailChimp or Mailgun for mailing,  Envato or Wix or Strikingly for making websites look nice, for providing social login.

But probably there are lots of people who don't want to, or don't know how to, do the development work required to integrate microservices.

It seems likely that another trend must follow, where Internet applications are offered that integrate microservices into a useful, cohesive whole.

Wednesday, April 29, 2015

Frustration and enlightenment: what learning new dev technology feels like.

Learning how to program is HARD.  Very hard.  

It's so hard that I imagine many beginners drop out because they think it's always going to stay hard.

But the good news is that if you stick with it, keep learning, keep trying and keep coding, then eventually you slowly accumulate enough understanding that the smoke of frustration clears and you start to become good at it, recognising problems, recognising ways of doing things, understanding how things are done.  There is a sense of enlightenment when eventually you cross that threshold from beginner to competent.  It's a great feeling.

But the thing that drives me mad is that I have to go back to being a beginner every time I need to learn a new technology.

After many cycles of learning new technologies I'm starting to recognise the process of being thrown back almost to beginner level each time I want to do something new.

Right now I am trying to learn "the right way" to do JavaScript web development using react.js and I've been thrown back to the intensely frustrating rank of "beginner". Everywhere there are mysteries and unexplained things.  Stuff doesn't make sense. Blog posts discuss how to get stuff done but make many assumptions about concepts that I haven't grasped yet - the author typically is so experienced that they aren't even aware that the reader might not grasp even the simplest of concepts need to understand the topic. A single sentence in a tutorial might hide an entire technology platform that I need to wrap my head around.

But I now recognise this frustrating learning process and know how to attack it. Here's what I do:

I read lots of basic tutorials, I don't try to deeply understand - too many pieces of the puzzle are missing, so I skim. 

I try to find some code examples that are very simple and very isolated and depend on no other tools or frameworks. 

I try to find complete, or larger applications that are "doing it the right way", get them to run and install,  poke around inside and see how the structure of a larger application holds together.

I might have had some specific questions or problems by this point so I post to the appropriate forum on reddit, or maybe email the author, or if I'm feeling masochistic, post a question to StackOverflow.

Essentially I aim to get one little thing to work - to make a chink in the armour of understanding. In this case for example I am aiming to get some data back from a web server and display it, and secondly to build an HTML form and POST it to the server.

I read the introduction documentation to how key related technologies work.  In this case obviously I have read introductions to react.js but it seems I also need to at least have a passing understanding of gulp and require and browserify and sass, so I fairly rapidly skim some intro documentation so I know how they fit into the picture and why they are there.

All during this process I am feeling frustrated and slow and unproductive. I have probably spent days achieving effectively nothing. If I was writing Python I would have been able to build almost a complete system. 

One of the challenging aspects of this type of learning is that at the beginning of the process it's not even necessarily clear what you are trying to do.  So along the way, as things become more clear, I form mini-goals and aim for them. In this case for example one of my mini-goals is to buy a react.js based template and get it running standalone on a web server (sound easy?  yes it should be, but for a beginner it's HARD).

After getting some small thing to work, probably one or two pieces of the puzzle are starting to fall into place, at least a little bit.  It's now that I go back to the beginning and repeat the cycle.  Right back to the start.  Re-read all those introductions, go back through all the code samples and re-do all the learning. In this second pass things should start to make alot more sense. Several lightbulbs should switch on. You will realise that the stuff you read previously already explained details that you missed, puzzles to solve, strange concepts and things you have been wondering about - it's just that the first time around you didn't have enough frame of reference to tie it all in to.

And all the time you need to be trying to get yourself in a position to write code.  Trying to get enough understanding to do a bit of coding.  Understanding only truly comes from coding with real purpose.  The objective of the reading and digging about is to get you to the code.

As mentioned before I like to write small isolated code samples at first, but ideally I am looking for a larger open source code base that already does something similar to what I am trying to do, and I want to work within it.  It's MUCH easier to grasp a new development technology from within a codebase that actually makes it work. Paradoxically it is also much harder because you have been dropped in the deep end, but that is a necessary part of the path of learning.  

When faced with learning a new technology I feel some sense of dread.  There's always a temptation to not learn the technology that you know would be the best one to know if the long run, but instead learn something simpler and easier.  Don't take that path - you'll just end up knowing the wrong technology and still not have moved towards deep understanding of the right technology.

So that is how I learn a new technology. It's frustrating and feels unproductive but deeply necessary.  To be a programmer in 2015 you need to competence with many types of technology and expertise with a few too. If you avoid this then you will not be able to get the stuff built that you need to. 

Even if you have attained enlightenment with some technology you will still be constantly repeating smaller iterations of this process as you learn new details about languages and frameworks, it never ends but hopefully becomes easier and faster.

You're going to need to repeat this learning process over and over and over so you may as well recognise the pattern of frustration through to enlightenment.  

It's VERY hard at first but it doesn't stay hard forever - work hard at getting through the initial beginner phase to competence.

It's the enlightenment stage that I look forward to - that's when programming becomes fun and the computer becomes your canvas upon which to paint your ideas and inspiration.

Tuesday, April 21, 2015

Millions of accounts are being compromised because developers don't have a specialised user database

One of the reasons that hacking incidents are so bad is because user data is stored in a normal database of some form - SQL, NoSQL, doesn't really matter which one, they are all unsuitable. Typically the user data sits in a table right alongside the rest of the application data. If a hacker gets access to the database machine or to the database query API then the hacker has unlimited access to download user data.  


This results in cases in which millions of user accounts are compromised. Another problem is that developers roll their own user and password management systems and get things like salting and hashing wrong, making the data vulnerable.

What developers need  is a minimal, single purpose database specifically designed for protecting user information and designed to move user data access away from the rest of the application data to minimise the impact of access by hackers.

Here are the requirements:
  • It should be accessible only via its specialised API which is designed to constrain the ways that it is accessed.
  • It should not provide generalised database query functionality. 
  • Its API should have password salting and hashing built in.
  • Its API should throttle access with some sort of algorithm designed to prevent downloads of large quantities of user data.
  • It should encrypt data internally.
  • It should communicate only over encrypted connections.
  • It should be distributed.
  • It should not be run on any web server, should run "behind the scenes" and be accessible only via its API.
  • It should include triggers and alerts based on uncommon access patterns or recognised nefarious access patterns.
  • It should have no other purpose.

Something like this wouldn't be a guarantee against being hacked but would be a good baseline for preventing common problems and minimising the outcomes of the seemingly inevitable hacks that we hear about all the time. Someone clever should write this.  I'd use it.

I bet there's someone out there smart enough to put this together in a matter of hours.

Monday, April 20, 2015

Startups take note - we're building the same damn thing over and over.

Got a new web based service you want to start selling? Well you're going to need to start coding the same damn thing yet again for the umpteenth time.  The same damn thing incidentally that the rest of the web development world is building over and over again.

Could some enterprising startup please stop this madness and build a generic web application that I can get going in 30 minutes with zero code?

Here's the components and requirements definining what is needs to do - easy to define because it's the same damn thing everyone is building. There are plenty of "best in class" companies in each category but none pull it all together into a seamless whole.  All assume that you're happy to start coding and integrating and getting bits from all over the place.

I want my users to be able to come to a gorgeous looking site (or at least clean and professional anyway) - must work well on mobile. The website is a very substantial part of the requirements. It should look like my website and not show any signs that it is actually constructed using something else. No "powered by Shopify" or anything on it.
Who does this part well: ThemeForest, Shopify, Strikingly, Wix, SquareSpace, TemplateMonster, Bootstrap

I want my users to be able to sign in and create an account using whatever the common signin services are (i.e. oauth)
Who does this part well:

Authorization and user management
I want the back end to handle all aspects of user registration and handle all the crap involved with sending out confirmations, password resets etc. the back end must provide me with an API. the back end must provide me tools for backing up my user and account data cause I don't trust that they won't go out of business one day the back end company must exude the sense that it's my customer base, not theirs. if I get the sense that I don't trust them then I'm outta there.
Who does some of this currently: StormPath, MailChimp

I want to be able to set up a simple price plan schedule of service levels and present it in that well known pricing table format that almost all websites seem to use.

MUST allow me to do subscription payments through Stripe - nothing else will do here.

Stripe have shown everyone how to do API's well. I want a incredibly thoroughly documented API with lots of practical examples and wide language support.

All the above must be nicely integrated, with ZERO coding required unless I choose to through the API.

Setup and configuration
Hard to see why it should take me more than half an hour in an admin interface to set up all the above.

See what I want? A beautiful, mobile ready site that my users can sign in to, create an account and buy my subscription service.  It should take me 30 minutes to hand over my credit card, choose a nice template, set up my pricing schedule and type in the text.

It's 2015 and I really shouldn't have to do three months coding to pull all this together. Neither should the rest of the development world because that's what a substantial number of developers are probably doing.

Give me an integrated, broadly featured up-and-running batteries included web based business in a box. I will pay because I value my time - I don't want to spend three months coding the same damn thing every time I want to launch a new product.

Tuesday, April 7, 2015

Slip of the finger, deleted 4 years and > 4,000 SMS messages. Thanks Apple.

iOS allows you to delete ALL SMS messages from a person by sliding their name and pressing the red button that appears.

Thanks Apple.  I wasn't trying to delete anything but accidentally slid the finger for whatever reason, saw a flash of the red button and bang, 4 or more years of SMS messages gone.  Goodness knows how many, probably more than 4,000.

You'd think the iPhone would say "are you sure you want to delete five years and 4,000 messages?" yes/no?

Yes theres a backup from a few months ago somewhere.  The messages aren't critical but it sure is annoying.

Thursday, April 2, 2015

The future: separate Internets with geographical borders

Eventually the Internet will divide. The cyberwars will lead countries to decide that it's not worth being connected to each other.

China will peel away to be an entirely separate Internet. There will be a Russian Internet. Others will separate.

The Internets won't be connected - it will be illegal, subverted by via secret, transient illegal connections.

Tuesday, March 31, 2015

Bare metal application builders are starting to flex their muscle.

As an application developer, security concerns me greatly. Every day there is a fresh announcement of a major hacking incident, often with the user database being compromised. I'd like to be able to develop applications that have some hope of resisting hackers.

One of the most promising ways to reduce the attack surface is to build applications that run on the bare metal of the CPU without a host operating system. More accurately, bare metal applications run directly on a virtualisation hypervisor rather than on the bare metal of a CPU a la MS-DOS.  The idea being that if there is no system to log in to then it's much harder for hackers to gain access. What can a hacker do if there's just nothing at all to log in to?

Running on bare metal is being made possible in large part by virtualisation technologies such as Xen which provide standard virtual networking and file system interfaces. These virtualised interfaces mean that bare metal solutions don't need hardware device driver support, making the core concept much easier to implement.

I'm super keen to be writing my applications to run on bare metal. There's quite a bit going on in this field but its early days; there's currently no practical way to write an application using mainstream programming languages and get it to run as bare metal. That's likely to change over the next couple of years. It is possible to build bare metal applications using  Ocaml, Haskell and Erlang.

I'd like to build bare metal applications using Python, Rust or Go. That's not possible right now. Here are the projects that I know about in the bare metal space:

MirageOS describes itself as "library operating system that constructs unikernels". The project appears to be relatively mature and actively developed. MirageOS requires that your code is written in OCaml.

Quoting from the website, HaLVM "enables developers to write high-level, lightweight virtual machines that can run directly on the Xen hypervisor". It is being developed by Galois, a U.S. company based in Portland, Oregon.  An overview presentation is here. You'll need to write your code in Haskell.

Ling allows Erlang applications to be run directly on the Xen hypervisor. It is being developed by Cloudozer, a startup company based out of Russia.

OSv appears to be a version of FreeBSD so severely stripped down that although applications still have the resources they need to run, there is little or no remaining recognisable operating system, for example there is no concept of users in OSv, and there are no drivers other than those required to run on a Xen hypervisor. OSv allows execution of JVM and Posix applications. OSv is developed by Cloudius Systems of Israel. A presentation is here.

Rump kernels
It appears that Rump kernels utilise NetBSD's user space device drivers to compile Posix applications into unikernels. Rumprun is an active project facilitating the build process. A recent tweet announced that MySQL has been built as a unikernel using Rumprun. Rumprun is described as "a wrapper for running programs that were written for a normal POSIX (NetBSD) system to run them under a rump kernel."

Determined not to leave the fun to the open source world, Microsoft Research has been doing some work on the library OS and its Drawbridge research project is described as "a research prototype of a new form of virtualization for application sandboxing. Drawbridge combines two core technologies: First, a picoprocess, which is a process-based isolation container with a minimal kernel API surface. Second, a library OS, which is a version of Windows enlightened to run efficiently within a picoprocess." Whether anything living will creep out of the bubbling green tubes of the research lab remains to be seen.

On the horizon.
The Cloudozer website mentions "Rust On Xen", "Go on Xen" and "J on Xen", exciting concepts but I couldn't find any further information. A recent post from Hajime Tazaki introduces a new project called LibOS, saying: "Our objective is to build the kernel network stack as a shared library that can be linked to by userspace programs to provide network stack personalization and testing facilities, and allow researchers to more easily simulate complex network topologies of linux routers/hosts." There's clearly more innovation and development in the works in the exciting field of bare metal application development. If you know of anything new, please let me know at

Hopefully it won't be too long until we see production quality ways to build bare metal applications using mainstream programming languages.

Monday, March 30, 2015

The continents of office and citizen computing are pulling apart.

Millions of years ago the earth had just one giant unified continent named Pangea. Over time, tectonic forces have pulled this continent apart.

For decades the worlds of office and citizen computing were unified. Visit just about anyone's house and you would see the citizen's preferred home computing environment looked very much like the office computing environment.

When most people go to work at an office, they sit down at a desktop computer for the day and that's how they get they job done. I'm using "desktop computer" loosely here - many office workers actually use laptops but the experience is effectively the same - it's a computer with a physical keyboard running Windows or OSX - general purpose operating systems.

Outside of the office world, people have moved strongly toward mobile devices - phones and tablets.

Seems to me no-one has given much thought to this growing difference between the preferred computing environment of the sit-down office world and the preferred computing environment of everyone else.

Will physical keyboards and general purpose operating systems always be the preferred computing environment in offices?

Are there unrealised opportunities to be found in the commitment of offices to desktop machines?

The continents of office and citizen computing are pulling apart. That change means opportunity for those who can see the new world through different eyes.

Thursday, March 26, 2015

Saturday, February 28, 2015

You'll pay massive interest on your technical debt - careful what you borrow from the future.

When you are programming, you incur a technical debt each time you chose to defer a task that is not absolutely required in order to get a working application.

The task probably still needs to be done at some point in the future, but you have saved time and effort in the short term by deferring it. Just as with money, technical debt requires you to pay in the future for the benefit you have taken in the present.

When you do come back later and see that task waiting to be done, it's going to cost much, much more time and effort to complete, compared to the time it would have taken you to do it in the first place.

When you first saw the task, you had your head in the code.  You had the context in your head of how it all fitted together, you had recently been thinking about the syntax, libraries and methods that this section of code uses.  You had some sense of how risky that code would be to build, and how it interacted with other parts of the system. You had the task context in your foreground thinking.

Whereas implementing the task in the first place might have taken you an hour, completing it as a technical debt task will require you to re-learn all that context.  It's going to take you ALOT more time to get all that context back into place and there's always a chance you won't be able to fully regain that context, or you'll miss something.  Chances are the price of your technical debt will be many times that of the initial time and effort required to complete that task.

Choose wisely which tasks you push into your technical debt, the interest rates can be staggering.

Sunday, February 22, 2015

How to get a job in programming: fix 200 bugs on a well known open source project.

This is a question that I am asked pretty much every day, now you have the answer.

End of lesson.

Thursday, February 12, 2015

Might it make sense to return from cloud hosted to self hosted servers?

Amazon EC2 was launched in 2006 and it was incredibly obvious what a good idea cloud computing was.

Up until then, to have a host computer on the Internet you needed to lug some big clunky box down to an Internet hosting data centre, where they ripped you off because there was so little competition.

Your big clunky box had one, two or more spinning fans in it. If (when) a spinning fan stopped, your machine died - you were in trouble.

Your big clunky box had a spinning hard disk in it. If (when) that stopped, your machine died - you were in trouble.

Your big clunky box power supply had to have huge capacity because CPUs chewed so much power. If (when) that stopped, your machine died - you were in trouble.

Your big clunky box was physically large - at least one full rack unit if not more would be taken up with what in todays terms is something pretty underpowered.

And when you were in trouble, you were in big trouble. You had to slouch off down to the data centre with your toolkit and spare machine and spend hours making the damn thing work again.

So when Amazon EC2 turned up it was screamingly obvious that cloud computing was a killer idea because it was just so much better than dedicated hosting in every possible way. Cloud computing as defined by EC2 was clearly one of the best ideas ever in technology. Hardware as software woo hoo.

BUT it's not 2006 anymore. Hardware is shrinking to the point of disappearing. Computers don't necessarily need CPU fans or power supply fans or 500watt power supplies or tower cases or even spinning hard disks. You can probably run a server in a data centre and have the reasonable expectation that it WON'T break any time soon, as opposed to 2006 in which you had the reasonable expectation that it WOULD break soon.

So I'm wondering, MAYBE, since 2006, the dedicated hosting data centre it has started to make more sense.

Maybe tiny cheap, highly reliable computers can be installed into your local data centre at very low cost and you can break free of the shackles, lock in and high prices (compared to owning your own hardware).

Maybe the world is different in 2015 and it might even be a good idea to start running your own computers again.

Just a thought. I think I'm go to go find the price lists for dedicated hosting at the local data centre.
The trends of server class computers becoming small and more reliable will continue into the future. The obvious issues of 2006 just aren't such big issues any more. Maybe it's time to break up the cloud and bring the servers back to homes, offices and data centres. That way at least you can see when the NSA is plugging in their USB monitoring devices, and offer their technicians a cup of tea while they work.

Wednesday, February 11, 2015

Structuring applications - Python SQLAlchemy

One of the things that interests me most is how to properly structure the source code of an application.

At the moment I am doing some stuff with SQLAlchemy so here is what I have found about how to correctly structure SQLAlchemy code over multiple modules/python files.

This from StackOverflow talks to the issue:

This from Michael Bayer, author of SQLAlchemy:!topic/sqlalchemy/BtPac9O3ggI