Thursday, April 30, 2015

Great software developers began non-technical & bulldozed obstacles to learning.

The coming micro service integration trend.

Currently there is a general trend towards Internet applications being very focused, minimalistic, sometimes even single-purpose. Examples include StormPath for user management, MailChimp or Mailgun for mailing,  Envato or Wix or Strikingly for making websites look nice, for providing social login.

But probably there are lots of people who don't want to, or don't know how to, do the development work required to integrate microservices.

It seems likely that another trend must follow, where Internet applications are offered that integrate microservices into a useful, cohesive whole.

Wednesday, April 29, 2015

Frustration and enlightenment: what learning new dev technology feels like.

Learning how to program is HARD.  Very hard.  

It's so hard that I imagine many beginners drop out because they think it's always going to stay hard.

But the good news is that if you stick with it, keep learning, keep trying and keep coding, then eventually you slowly accumulate enough understanding that the smoke of frustration clears and you start to become good at it, recognising problems, recognising ways of doing things, understanding how things are done.  There is a sense of enlightenment when eventually you cross that threshold from beginner to competent.  It's a great feeling.

But the thing that drives me mad is that I have to go back to being a beginner every time I need to learn a new technology.

After many cycles of learning new technologies I'm starting to recognise the process of being thrown back almost to beginner level each time I want to do something new.

Right now I am trying to learn "the right way" to do JavaScript web development using react.js and I've been thrown back to the intensely frustrating rank of "beginner". Everywhere there are mysteries and unexplained things.  Stuff doesn't make sense. Blog posts discuss how to get stuff done but make many assumptions about concepts that I haven't grasped yet - the author typically is so experienced that they aren't even aware that the reader might not grasp even the simplest of concepts need to understand the topic. A single sentence in a tutorial might hide an entire technology platform that I need to wrap my head around.

But I now recognise this frustrating learning process and know how to attack it. Here's what I do:

I read lots of basic tutorials, I don't try to deeply understand - too many pieces of the puzzle are missing, so I skim. 

I try to find some code examples that are very simple and very isolated and depend on no other tools or frameworks. 

I try to find complete, or larger applications that are "doing it the right way", get them to run and install,  poke around inside and see how the structure of a larger application holds together.

I might have had some specific questions or problems by this point so I post to the appropriate forum on reddit, or maybe email the author, or if I'm feeling masochistic, post a question to StackOverflow.

Essentially I aim to get one little thing to work - to make a chink in the armour of understanding. In this case for example I am aiming to get some data back from a web server and display it, and secondly to build an HTML form and POST it to the server.

I read the introduction documentation to how key related technologies work.  In this case obviously I have read introductions to react.js but it seems I also need to at least have a passing understanding of gulp and require and browserify and sass, so I fairly rapidly skim some intro documentation so I know how they fit into the picture and why they are there.

All during this process I am feeling frustrated and slow and unproductive. I have probably spent days achieving effectively nothing. If I was writing Python I would have been able to build almost a complete system. 

One of the challenging aspects of this type of learning is that at the beginning of the process it's not even necessarily clear what you are trying to do.  So along the way, as things become more clear, I form mini-goals and aim for them. In this case for example one of my mini-goals is to buy a react.js based template and get it running standalone on a web server (sound easy?  yes it should be, but for a beginner it's HARD).

After getting some small thing to work, probably one or two pieces of the puzzle are starting to fall into place, at least a little bit.  It's now that I go back to the beginning and repeat the cycle.  Right back to the start.  Re-read all those introductions, go back through all the code samples and re-do all the learning. In this second pass things should start to make alot more sense. Several lightbulbs should switch on. You will realise that the stuff you read previously already explained details that you missed, puzzles to solve, strange concepts and things you have been wondering about - it's just that the first time around you didn't have enough frame of reference to tie it all in to.

And all the time you need to be trying to get yourself in a position to write code.  Trying to get enough understanding to do a bit of coding.  Understanding only truly comes from coding with real purpose.  The objective of the reading and digging about is to get you to the code.

As mentioned before I like to write small isolated code samples at first, but ideally I am looking for a larger open source code base that already does something similar to what I am trying to do, and I want to work within it.  It's MUCH easier to grasp a new development technology from within a codebase that actually makes it work. Paradoxically it is also much harder because you have been dropped in the deep end, but that is a necessary part of the path of learning.  

When faced with learning a new technology I feel some sense of dread.  There's always a temptation to not learn the technology that you know would be the best one to know if the long run, but instead learn something simpler and easier.  Don't take that path - you'll just end up knowing the wrong technology and still not have moved towards deep understanding of the right technology.

So that is how I learn a new technology. It's frustrating and feels unproductive but deeply necessary.  To be a programmer in 2015 you need to competence with many types of technology and expertise with a few too. If you avoid this then you will not be able to get the stuff built that you need to. 

Even if you have attained enlightenment with some technology you will still be constantly repeating smaller iterations of this process as you learn new details about languages and frameworks, it never ends but hopefully becomes easier and faster.

You're going to need to repeat this learning process over and over and over so you may as well recognise the pattern of frustration through to enlightenment.  

It's VERY hard at first but it doesn't stay hard forever - work hard at getting through the initial beginner phase to competence.

It's the enlightenment stage that I look forward to - that's when programming becomes fun and the computer becomes your canvas upon which to paint your ideas and inspiration.

Tuesday, April 21, 2015

Millions of accounts are being compromised because developers don't have a specialised user database

One of the reasons that hacking incidents are so bad is because user data is stored in a normal database of some form - SQL, NoSQL, doesn't really matter which one, they are all unsuitable. Typically the user data sits in a table right alongside the rest of the application data. If a hacker gets access to the database machine or to the database query API then the hacker has unlimited access to download user data.  


This results in cases in which millions of user accounts are compromised. Another problem is that developers roll their own user and password management systems and get things like salting and hashing wrong, making the data vulnerable.

What developers need  is a minimal, single purpose database specifically designed for protecting user information and designed to move user data access away from the rest of the application data to minimise the impact of access by hackers.

Here are the requirements:
  • It should be accessible only via its specialised API which is designed to constrain the ways that it is accessed.
  • It should not provide generalised database query functionality. 
  • Its API should have password salting and hashing built in.
  • Its API should throttle access with some sort of algorithm designed to prevent downloads of large quantities of user data.
  • It should encrypt data internally.
  • It should communicate only over encrypted connections.
  • It should be distributed.
  • It should not be run on any web server, should run "behind the scenes" and be accessible only via its API.
  • It should include triggers and alerts based on uncommon access patterns or recognised nefarious access patterns.
  • It should have no other purpose.

Something like this wouldn't be a guarantee against being hacked but would be a good baseline for preventing common problems and minimising the outcomes of the seemingly inevitable hacks that we hear about all the time. Someone clever should write this.  I'd use it.

I bet there's someone out there smart enough to put this together in a matter of hours.

Monday, April 20, 2015

Startups take note - we're building the same damn thing over and over.

Got a new web based service you want to start selling? Well you're going to need to start coding the same damn thing yet again for the umpteenth time.  The same damn thing incidentally that the rest of the web development world is building over and over again.

Could some enterprising startup please stop this madness and build a generic web application that I can get going in 30 minutes with zero code?

Here's the components and requirements definining what is needs to do - easy to define because it's the same damn thing everyone is building. There are plenty of "best in class" companies in each category but none pull it all together into a seamless whole.  All assume that you're happy to start coding and integrating and getting bits from all over the place.

I want my users to be able to come to a gorgeous looking site (or at least clean and professional anyway) - must work well on mobile. The website is a very substantial part of the requirements. It should look like my website and not show any signs that it is actually constructed using something else. No "powered by Shopify" or anything on it.
Who does this part well: ThemeForest, Shopify, Strikingly, Wix, SquareSpace, TemplateMonster, Bootstrap

I want my users to be able to sign in and create an account using whatever the common signin services are (i.e. oauth)
Who does this part well:

Authorization and user management
I want the back end to handle all aspects of user registration and handle all the crap involved with sending out confirmations, password resets etc. the back end must provide me with an API. the back end must provide me tools for backing up my user and account data cause I don't trust that they won't go out of business one day the back end company must exude the sense that it's my customer base, not theirs. if I get the sense that I don't trust them then I'm outta there.
Who does some of this currently: StormPath, MailChimp

I want to be able to set up a simple price plan schedule of service levels and present it in that well known pricing table format that almost all websites seem to use.

MUST allow me to do subscription payments through Stripe - nothing else will do here.

Stripe have shown everyone how to do API's well. I want a incredibly thoroughly documented API with lots of practical examples and wide language support.

All the above must be nicely integrated, with ZERO coding required unless I choose to through the API.

Setup and configuration
Hard to see why it should take me more than half an hour in an admin interface to set up all the above.

See what I want? A beautiful, mobile ready site that my users can sign in to, create an account and buy my subscription service.  It should take me 30 minutes to hand over my credit card, choose a nice template, set up my pricing schedule and type in the text.

It's 2015 and I really shouldn't have to do three months coding to pull all this together. Neither should the rest of the development world because that's what a substantial number of developers are probably doing.

Give me an integrated, broadly featured up-and-running batteries included web based business in a box. I will pay because I value my time - I don't want to spend three months coding the same damn thing every time I want to launch a new product.

Tuesday, April 7, 2015

Slip of the finger, deleted 4 years and > 4,000 SMS messages. Thanks Apple.

iOS allows you to delete ALL SMS messages from a person by sliding their name and pressing the red button that appears.

Thanks Apple.  I wasn't trying to delete anything but accidentally slid the finger for whatever reason, saw a flash of the red button and bang, 4 or more years of SMS messages gone.  Goodness knows how many, probably more than 4,000.

You'd think the iPhone would say "are you sure you want to delete five years and 4,000 messages?" yes/no?

Yes theres a backup from a few months ago somewhere.  The messages aren't critical but it sure is annoying.

Thursday, April 2, 2015

The future: separate Internets with geographical borders

Eventually the Internet will divide. The cyberwars will lead countries to decide that it's not worth being connected to each other.

China will peel away to be an entirely separate Internet. There will be a Russian Internet. Others will separate.

The Internets won't be connected - it will be illegal, subverted by via secret, transient illegal connections.