KalyanChakravarthy.net

Thoughts, stories and ideas.

Python Deployments Wishlist

Am currently faced with a baffling realisation of the reality that there is no one sane way to deploy python applications.

It's utterly annoying that I cannot just tar a virtual env and use it.
Performing pip install on every deploy is plain ridiculous. Downloading from the internet on every code change with separate virtualenv is a nogo. No RPMs, nope, no way am writing another spec file.

Wishlist:

  • Deployment tooling that allows me to go from zero to production with a new application with no setup.
  • No-downtime between deployments (has to work with a single instance)
  • Philosophy of implementations should make sense whether deploying to one or a 100 servers.
  • Controlled activation of the new version of software i.e. traffic shifting.
  • Stable re-producible environment setup everytime (perhaps with ansible).
  • Cli that lets you bootstrap, build, deploy and control activation of software (nginx or via LB).

Next:
Am starting to hack a prototype with fabric, python and Ansible, which is already proving to be challenging but looks doable.

Bug in my code and Ballet

... of sorts, or more accurately barre.

I use a service called Onefit in the Netherlands which lets me register for group lessons or use a gym in a bunch of places (limit is four times a month). I used it for joining yoga classes sometime back. Apparently its quite popular with locals here and so I have a hard time finding a spot. So like any sensible person, I tried to automate it. I wrote a python script that would run in the background and try to find me a spot when someone cancels their's (and people do cancel as there is a no-show fee). Lo and behold the script alerts me, and I immediately sign up. Now am feeling excited that I am the smartest person around to do such a thing.

The day of the supposed yoga class, am all pumped up to be going back after 2-3 months. I get to the venue on time and tell the person at the reception that I am here for the yoga class. One of the persons sitting there was the instructor, and she thought I was confused, so she mentioned there is no yoga class for another hour. I tell her, no, look and show her the confirmation on my phone (there is a location-based check-in), and she went "yeah no, its a 'Barre' class" which is a Ballet style workout.

Now am visibly shaken, my mind wanders away, slowly, mentally debugging the code that I refactored day earlier, as to what just happened, all the while still blankly looking at that lady. It turns out I applied filters for location type, time but not for the class type. Neither during my testing did I realise that they were offering more variety of classes from the last time I tried months earlier, nor when I was pressing the "book" button. I can at this point see the code and what the error was.

What do I do now?

It is too late. I had to do something, either go forward or back home. One on hand I begin to wonder if will I be charged a cancellation fee? On the other hand, I asked myself, would I have ever tried this by my own volition? The cancellation fee of €10 would be a blow to my ego that thought it could automate itself out to glory.

So, I went in. Yes. It happened.

Boy oh boy, what an experience it was. I was the only guy and a chubby human in the entire class, full of women in shape and active. Halfway through, am sweating, panting, full of feeling that I can collapse any moment. I keep going, and then I could no more and had to take a timeout. And all throughout, everyone was going through the motions of the class without breaking a sweat, like it was some fucking music video. What the hell.

Usually when I play sports, its the next day I feel it. No sooner than the class ended, I could feel my body telling me what a dumb idea that was.

And gents, that's how I ended up trying ballet workout, all due to a bug.
So, remember the lesson kids, test your code :)

PS: Now this made me wonder about something a friend said a long back, during one of our conversations about "anti-search" search engines, which recommend something that you wouldn't usually try, either a thing or a place, a sight or an experience. While I have tried random things in the past, if not for this "bug", don't' think I would have intentionally tried to go check this one out. Serendipity can be a good thing, and I wish this world facilitated more of it.

People

Ever since we are born into this world, our lives begin by being cared by others. We grow to realise, implicitly that people are essential. How could we comprehend any other possibility?

So we try to be friendly, often more than that is necessary or possible to be reciprocated as humans are different. Why do it then? Because it's convenient to think, to have more friends is to share good times with many, thereby meaning more happiness.

The danger of it is letting toxic people in. They will come, eventually. They always do. The trick is to recognise who are the ones causing the sort of drama which leaves you not feeling at the top of the world.

People who cause drama, need to be weeded out, ruthlessly. Those blissfully unaware of the damage it does to those around may perhaps deserve the benefit of a doubt, once. But just once. Life is too short to try to be nice towards those adding misery.

Having fewer friends who are more thoughtful and caring is not only energising, but also adds an immense amount of meaning to life. The value of a meaningful relationship is unmatched against a thousand casual friendships or a hundred one night stands.

Books in 2017

Last year I read quite some books, unconsciously, without much thought but entirely engrossed. It bought me peace and quite. A big chunk of it was fiction.

As a challenge of not only equalling the amount of reading I get done, I also set myself up to expand on the topics to read. This has been hard so far.

Here is a list in reverse chronological order that I finished reading. I read 30 in all. 2 of which are audio books.

1. The Ministry of Utmost Happiness

Arundhati Roy

Notes: Heart touching, soul-crushing, nostalgia-inducing, poetry filled and an ecstasy evoking read.

Fiction straying not far from facts, blurred the lines between reality and imagined. The vivid descriptions of Delhi, J&K, the people and everything about India, has left me yearning to see them for myself in the new light shone by Arundhati Roy through her words

2. Letters to a Young Poet

Rainer Maria Rilke

3. Blind Man's Bluff

Sherry Sontag
Notes: I always enjoy military non-fiction. This did not disapoint although it didn't match up to my expectations of submarine warfare set with the Thunder Below: The USS Barb book

4. The Songs Of Distant Earth

Arthur C. Clarke

5. Roadside Picnic

Arkady Strugatsky

6. A Chinese Life

Li Kunwu

7. Body Language

Glenn Wilson

8. Messy: How to Be Creative and Resilient in a Tidy-Minded World

Tim Harford

9. Outliers: The Story of Success

Malcolm Gladwell

10. Production-Ready Microservices

Susan J. Fowler

11. The Dispatcher

John Scalzi

12. Rogue Heroes: The History of the SAS

Ben Macintyre

13. Consider Phlebas (Culture, #1)

Iain M. Banks

14. The Power of Habit: Why We Do What We Do in Life and Business

Charles Duhigg
Notes: Very fascinating

15. Night School (Jack Reacher #21)

Lee Child

16. Norse Mythology

Neil Gaiman

17. Make Your Bed

William H. McRaven

18. Command and Control

Eric Schlosser

19. The Subtle Art of Not Giving a F*ck

Mark Manson

20. Prisoners of Geography

Tim Marshall
First forray into politics and current affairs. I would love to read more of such books.

21. A short history of nearly everything

Bill Bryson
Notes: Literally tid bits of everything that lead to modern world.

22. Skunk Works

Ben R. Rich
Notes: Nonfiction, Science, Spy planes, Engineering.

23. 2001: A Space Odyssey (Space Odyssey #1)

Arthur C. Clarke
Notes: Classic. Reminded me so much about Interstellarthe movie andRendezvous with rama

24. The End of All Things (Old Man's War #6)

John Scalzi

25. The Human Division

John Scalzi

26. Zoe's Tale (Old Man's War, #4)

John Scalzi

27. Agent M: The Lives and Spies of MI5's Maxwell Knight

Henry Hemming
Notes: Feels like a spy novel. Hard to believe it is nonfiction.

28. Dune (Dune Chronicles, #1)

Frank Herbert
Notes: different science fiction.

29. Lords of the Sky: Fighter Pilots and Air Combat, from the Red Baron to the F-16

Dan Hampton
Notes: Fighter jets, their evolution, the role they played in conflicts around the world from before WW1, through WW2, Vietnam war, Desert Storm (Iraq) and further.

30. The Quantum Thief (Jean le Flambeur, #1)

Hannu Rajaniemi
Notes: Different science fiction. Set in a world where souls can exist, thoughts and memories can be shared by means of mental contracts. Mind bending.

Introduction to R language

Inspired by a friend's prolific use of R for doing data analysis, over past few days I have been trying to get the hang of the language. Here are some notes and thoughts

First impressions

Thinking of R as a programming language may sub-consciously invite a notion of procedural logic. That was a path not worth going down. Its easier to assimilate the syntax, if one were to consider it an expression evaluation system.

Not having worked with any kind functional languages (unless you consider javascript), I had to find analogies to the procedural world, to help grasp the fundamentals.

Here are my notes starting from the very basics to the point of being able to make a basic plot.

Docs

Am following documentation here - https://cran.r-project.org/doc/manuals/R-intro.html. For the most part, it is great, although some parts may seem tediously verbose, fretting on about smallest details.

Getting Started - IDE

R Studio is a pretty good place to start learning and using R. It features.

  • REPL (interactive prompt)
  • Notebook from which you can execute one or many statements into your REPL.
  • package manager
  • Graphics/output viewer
  • Variable viewer

REPL

Python's REPL allowed me to learn the language without leaving the convenience of the terminal and in the absence of internet a decade ago in India.

R's REPL in R-Studio takes it a notch further.

Getting help is quite easy. Simple executing ?func will open docs for that function.

> ?plot

Types

The language is loosely typed. Variables can be assigned to one type and re-assigned to another later.

> a = "foo"
> class(a)
[1] "character"
> a = 23
> class(a)
[1] "numeric"

R being primarily a statistical analysis language, the primitive data types are quite diverse. A non-exhaustive list for an idea:

  • numeric
  • complex
  • character (strings)
  • vectors (think arrays)
  • matrices
  • factors
  • data frames (think tables)
  • etc

Assignment

Variables are assigned from right to left with either = or <-. You can also perform left to right assignments by the use of -> operator.

> a <- 23**2
> 23**2 -> a # assign expression into a
> print(a)
[1] 529

Vectors

Closest (and insufficient) analogy is to think of them as arrays, but richer.

create vectors

A vector can be initialised using c()

> c(1,123,33,223,2) -> a
> print(a)
[1]   1 123  33 223   2
> sum(a)
[1] 382

All elements of the vector must be of the same type. If elements all but one are numeric and last is string, all are coerced to become a string

> c(1,2,3,"f")      # all but one are numeric
[1] "1" "2" "3" "f" # all coerced to strings
> c(1,2,3,"f") -> a
> class(a)
[1] "character"

c() function flattens nested vectors

> c(a, c(1,2,3))
[1] "1" "2" "3" "f" "1" "2" "3"

length and indexing

Length can be computed using length() function.

Items can be accessed using a one-index notation.

> a
[1] "1" "2" "3" "f"
> a[1]
"1"

range of elements

Range of items can be expressed as variable[from: length]

> a
[1] "1" "2" "3" "f"
> a[0:2]
[1] "1" "2"

Numeric vectors can also be created using the range syntax

> 1:10
 [1]  1  2  3  4  5  6  7  8  9 10

out of bounds

Index can go out of bounds. Out of bounds elements will be marked with a typed NA value.

> a
[1] "1" "2" "3" "f"
> a[0:10]
[1] "1" "2" "3" "f" NA  NA  NA  NA  NA  NA

expression selection

a[<expression>] can contain any valid expression and is not limited to range representation

> b = c(1,2,3,NA,NA) # consider
> b[b>2]             # elements of b > 2
[1]  3 NA NA

> b[ !is.na(b) ]     # elements of b that is not `NA`
[1] 1 2 3

> b[ b > 2 & !is.na(b) ] # b > 2 and that is not `NA`
[1] 3

using operators

Any basic math operation performed, is applied to whole vector and result is represented as an equivalent vector.

> b = c(1,2,3)   # assign
> b
[1] 1 2 3
> b + 1          # +1 all elements
[1] 2 3 4
> b*b            # square all elements
[1] 1 4 9

Addition of multiple vectors will result in one-to-one addition.

> c <- 1:5                  # c = c(1,2,3,4,5)
> d <- 6:10                 # d = c(6,7,8,9,10)
> c + d                     
[1]  7  9 11 13 15          # 1+5, 2+6

vectors of in-equal can participate in binary operations - the catch being the shortest vector will be repeated

> 1:6 -> p        # 1 2 3 4 5 6
> 1:3 -> q        # 1 2 3
> p + q           
[1] 2 4 6 5 7 9   # 1+1, 2+2, 3+3, 4+1, 5+2, 6+3

labels

Elements of a vector can be named

> x = c(foo=1,2,3,4,5)  # named vector element
> x
foo                     # column name
  1   2   3   4   5 

It can be accessed both by an index as well as by its name.

> x[1]
foo 
  1 
> x["foo"]
foo 
  1 

The names can be obtained by names function.

> names(x)
[1] "foo" ""    ""    ""    ""

Plotting a simple vector

Consider vectors

> x = 1:5
> x
[1] 1 2 3 4 5
> x^3
[1]   1   8  27  64 125

Plotting x vs x^3 is as simple as running the function plot()

> plot(x = x, y = x**3, type='b')

Note on the parameter type:

  • 'b' - indicates "both" points and lines
  • 'p' - will plot only points
  • 'l' - will plot only lines