Tag Archives: experience

Python Technical Interview – An Experience

As a freelancer one of the things that comes with getting a project/job is handling technical interviews. I have so far managed to convince the client with a work sample, test project …etc., This is literally the first time I sat for a full technical interview. And it did teach a few lessons. Let me document it for future use.

It started off with the basic of the language:

1. What is the difference between an iterable and an iterator?

Vincent Driessen provides a clear explanation of the difference with the examples here https://nvie.com/posts/iterators-vs-generators/

As an aside, he has a number of posts which are really great like his Git workflow model that I have used in my projects. Bookmark it

2. What is a Context Manager? What is its purpose? How is it different from a try…finally block? Why would you use one over another?

Context Manager are functions/classes that allow us to allocate and release resources as required. Used with the with keyword in code.

The difference between context manager and try..finally block is explained in technical detail here: https://stackoverflow.com/questions/26096435/is-python-with-statement-exactly-equivalent-to-a-try-except-finally-bloc

But a simpler more practical difference is given by Dan Bader: https://dbader.org/blog/python-context-managers-and-with-statement

3. Can you tell me some advantages of Python over other languages?

I rambled something like, it is is easier to read and write. The file structure (I should have said modules/packages) is great. Even modern iterations of Javascript are copying the import from syntax. Native implementation of a lot of things in standard library…etc.,et.,

But the thing my interviewer was looking for were the words “automatic garbage collection” because the next question was

4. How does Python handle memory?

Python has automated memory management and garbage collection.That is why we never worry about how much memory we are allocating like C’s malloc `calloc functions.

5. Do you know how Python does that? Do you know about GIL?

sheepish smiles and saying no’s ensued. I ran into an issue a few months back, I think maybe with a DB connection issue or something which led me on a rabbit hole that ended with GIL. I should have learnt it that day.

Anyway, here is the article about Python’s memory management. https://realpython.com/python-memory-management/

6. Have you worked on projects involving multi-threading? What do you know about multi-threading?

I hadn’t. Someday maybe I will.

7. Can you explain in detail the steps involved in a form submit to response cycle in detail?

https://developer.mozilla.org/en-US/docs/Learn/HTML/Forms/Sending_and_retrieving_form_data

8. How does the browser know where your server is when the information is submitted to a particular URL?

DNS servers – IP resolution

9. The server sends back text as a string how do you see colorful information in browser?

The text is converted into DOM elements which are rendered by the browsers rendering engine.

10. If a browser is showing unreadable character and question marks instead of displaying the information what could be the reason?

Document Encoding mismatch. The server might send the data encoded in Unicode UTF-8 and the browser might be decoding it as ASCII or LATIN-1 resulting in weird characters and question marks being rendered in the browser.

11. You said Unicode and UTF-8 what is the difference?

Unicode is the term used to describe the character set. If it is encoded with 8 bits it is called UTF-8, if encoded with 16 bits it is called UTF-16 etc.,

For deep dive into Unicode (a must): https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses/

12. What kind of request does the browser make to a server? And what are the types of requests that can be made?

Browsers make a HTTP requests. The types are GET, POST, PUT, DELETE, HEAD, OPTIONS ..etc., (I think I said UPDATE instead of PUT, silly)

13. What is the difference between `==` and `===` in JavaScript?

StackOverflow: https://stackoverflow.com/questions/523643/difference-between-and-in-javascript

Some other questions, that were asked:
1. Do you know Docker? Have you used AWS?
2. Do you know Data Base schema design?
3. You have a SQL query that takes a long time to execute. How would you begin to make it faster? Do you know about Query optimisation and execution plans?

Sensible Test Data

I am currently working on a project called Peer Feedback, where we are trying to build a nice peer feedback system for college students. We use Canvas Learning Management System (CanvasLMS) API as the data source for our application. All data about the students, courses, assignments, submissions are all fetched from CanvasLMS. The application is written in Python Flask.

Current Setup

We are mostly getting data from API, realying it to frontend or storing it in the DB. So most of our testing is just mocking network calls and asserting response codes. There are only a few functions that contain original logic. So our test suite is focused on those functions and endpoints for the most part.

We recently ran into a situation where we needed to test something that involved the fetching and filtering data from API and retriving data from DB based on the result.

Faker Library and issues

The problem we ran into is, we can’t test the function without first initalizing the database. The code we had for initializing the CanvaLMS used the Faker library, which provides nice fake data to create real world feel for us. But it came with its own set of problems:

Painful Manual Testing

While we had the feel of testing realworld information, it came with real world problems. For e.g., I cannot log in as a user without first looking up the username in the output generated during initialization. So I had to maintian a postit on my desktop, use search functionality to find the user I want to test and copy his email and login with it.

sensible_test_data_1

Inconsistency across test cycles

When we write our tests, there is no assurity that we could reference a particular user in the test with the id and expect the parameters like email or user name to match. With the test data being generated with different fake data, any referencing or associaation of values held true only for that cycle. For e.g, a function called get_user_by_email couldn’t be tested because we didn’t know what to expect in the resulting user object.

Complex Test Suite

To compensate for the inconsistency in the data across cycles, we increased the complexity of the test suite, we saved test data in JSON files and used it for validation. It became a multi step process and almost an application on its own. For e.g, the get_user_by_email function would first initialize the DB, then read a json file containing the test data and get a user with email and validate the function, then find the user without an email and validate it throws the right error, find the user with a malformed email… you get the idea. The test function itself not had enough logic warranting a test of test suites.

Realworld problems

With the realworld like data came the realworld problems. The emails generated by faker are not really fake. There is a high chance a number of them are used by real people. So guess what would have happened when we decided to test our email program 🙂

Sensible Test Data

We finally are switching to a more sensible test data for our testing. We are dropping the faker data for user generation, and shifiting to the sequencial user generation system with usernames user001 and emails user001@example.edu. This solves the above mentioned issues:

  1. Now I can login without having to first look it up in a table. All I need to do is append a integer to the word user
  2. I can be sure that user001 would have the email user001@example.edu and that these associations will be consistent across test cycles.
  3. I no longer have to read a JSON file to get a user object and test related information. I can simply pick one using the userXXX template, reducing complexity of the test suite.
  4. And we won’t be getting emails for random people asking us to remove them from mailing lists and probably we are saving ourselves being blacklisted as a spam domain.

Conclusion

Faker provided us with data which helped us test a number of things in the frontend, like the different length in names, multi part names, unique names for testing filtering and searching etc., while also added a set of problems that makes our work difficult and slow.

Our solution to having a sensible dataset for testing is plain numerically sequenced dataset.

Update

Using the generic name tags like user was still causing friction as we have multiple roles like teacher, TAs, students …etc., So I improved it further by creating users like student0000, ta000, `teacher00“.

Facebook Account Deletion

I just now deleted my Facebook account.

Why?

  1. Freedom – Day by day using it makes me feel as if someone is dictating me how and what I should be talking to people about. The rampant, out of the blue censors by law enforcement and judiciary are the reasons. Even though I have never been a part of any such action, the atmosphere is getting too toxic over there in that domain.
  2. Irrelevancy – Most of the content that I encounter on my timeline are completely irrelevant to me and the only content that I engage with are blogs like Lifehacker and posts of a set of very few friends, both of which I think I can do without having to endure a non-free environment.
  3. Privacy – It has been a great concern for me ever since I became aware of its importance and ethics. Somehow kept telling myself, nothing would affect me. I think underestimating risks is something I am unwilling to do these days after reading “Fooled by Randomness“.
  4. Paranoia – Closely connected to “Privacy” – I was always thinking what if someday I discover that all the stuff that I have shared is sold/shared to someone and that someone uses it for purposes I never suppose it to be? Am I planning murder? No. But what if someone frames me for it? I am NOT paranoid by nature, I walk 1 among the billion+ in the country. But again, not all billion+ walk around recording data about what they read, where they go and what they think.
  5. Narcissism – One of the biggest effect of Facebook on character is, I think, breeding narcissism. It breeds a kind of self importance and provides an easy sense of achievement and forces a person to project a personality, real or otherwise. This becomes especially stressful when considering the fact when one is connected to all sorts of people from high school friends to working colleagues. A few recent encounters with people has left a bad taste in  mouth about the whole “sharing” thing.
  6. Spam – Spam, spam, spam – there is just no end to it. I am generally very efficient in ignoring Ads and spam because all that I look for is content from people and never click any link other than blog-posts. But most of the time genuine and original content is very hard to come by and all sorts of false claims on history, technology, identity creep in along with celebrity, movie and all sorts of eye-ball catching stuff.

After considering all this, the most logical decision to make seems to be to leave the system and reduce a lot of responses that brain generates due to unnecessary stimuli.

Will I ever get on a social network again?

I currently do not have an answer to that. There are a lot of things to be considered as listed above and see how they play out to make a decision. With the current turmoil of internet censorship across the world, misuse of it as tools of destruction, authoritative control, capitalist bait holder, and as an enterprise looking to make money indirectly by using my data, I don’t think I am getting onto it (FB) any time soon. I specifically mean FB here because, there exists an alternate platform called Diaspora* which deals with all of my above concerns (okay, maybe not narcissism), but  jumping into such a platform is pointless if I don’t really have the people I want to interact.

The Setup

Continuing from the previous post, let me write down everything that defined my work setup.

The Curriculum

This is where the fun starts. I worked with two different curriculum throughout the year. The school wanted me to teach the mandated Samacheer syllabus and the organisation that I work for wanted me to teach to the Common Core Standards. The Samacheer part contained the Social, Science and the “English as a subject” subjects to be taught, the organisation gave me Mathematics  and “English as a standard” subjects to teach. It is actually painful to be a teacher and teach language either as a subject or as a set of standards. More on that separately sometime later (which is almost never).

The Red Ink

There were 37 notebooks for each (3) Samacheer subject to be checked and corrected 3-5 times in every term. Each term itself is about 3 or 4 months. And there where 2 mid-term term tests and 1 end of term test for each term. For the organization’s part, we were supposed to conduct Unit Assessments, which is one in 6 weeks, Weekly assessments and if possible Daily Assessments. I just did the Unit Assessments. Tried Weekly assessments but dropped it after a couple of weeks, it was getting out of hands. English made up for it, by making me correct a set of at least 10 questions every alternate day. I remember sitting, standing, sleeping, walking and even jumping on/off trains with my bag on the shoulder, papers on the left hand and red pen on the right.

The Sessions

The organisation’s way of making sure we are fully equipped to handle everything in the classroom. It was usually planned in the evenings after school when we are in our lowest glucose levels and looking out for a corner to curl. The sessions did make a lot of sense to the people who were organizing them. They were usually about how to teach, how to handle kids, how to understand a particular area in order to deliver it the way it is supposed to be. But one thing no one seemed to care/understand/grasp was there was no single way to do stuff.

The Printer

Canon LBP2900. One trademark of being a TFI fellow is we print more paper for each kid than what government or the school would. Having a laser printer really does help. One can be free of the timing restrictions imposed by the Xerox shops and save a lot more money. I printed about 8000-9000 pages in the last 4 months alone. 1500 rupees for all that paper and 400 rupees for the toner and the immense flexibility of being able to print whatever and whenever.

The Travel

The travel was two/three legged. I usually started off with short bus ride 5E/23C/49 from Adyar depot to Madhya Kailash, took a train from Kasthuribai Nagar station to the Beach Station, and then finally took 44C from Beach Station to the Power House stop. Sometimes the 5E-Train combo was replaced by the 21H/PP19 from Adyar Depot to Parry’s Corner. Initially used 6D from the backside of Adyar Depot, but extra 300m walking and having no alternate buses made me switch to other options. One thing good about the train travel is I always found space to sit and even work on the laptop if required. Having a monthly season ticket for just 105 rupees was another boon. Never had to worry about tickets/queues and oversleeping during return journeys.

 

These define the physical boundaries of how I worked in the past one year. But how did I actually work? What was “The process”?

The Idea of Democracy

Before I begin, (can be skipped)

I have a few things going inside my brain as if spun like the saw dust in convection experiment. Though mostly ignored, these things tend to jump in between a conversation or tend to pull conversations towards them so that they can show themselves. This has resulted in a number of “long pause” moments in my conversation. This poor blog is the place where I decided to pour them down, so this process of convection can stop.

The Scene

Most Indians would have had “democracy” explained to them them, as such, in their schools via the standard definition of “by,of,for the people”. What it means in real life, though is understood later in life when circumstances set right. I understood it when I watched a 70s, probably even 50s or 60s, movie. I don’t remember the entire story line, but the scene which gave me idea of democracy sticks to this moment. The scene unfolds as follows:

A group of rebels fight for the freedom and liberation of their land from the clutches of the king, they call evil and his rule tyranny, and establish democracy. On a particular day, they get hold of a bodyguard of the king. He is brought in chains and made to stand before the council of the rebel leaders, who are sitting over a table, discussing strategies. The bodyguard reveals nothing to the council’s questions and proclaims his loyalty to the king. The head of the council tells his men to throw him in a dark cell. As the men holding the chain wait for the council members to leave before they can take the bodyguard away, a man brings dinner to the leader and places it on the table and spills the drink on him. The leader becomes furious at this indignation in front of a enemy and beats the man (a servant in his mind). At this point, the bodyguard laughs aloud and asks:

“Is this what you call democracy? Is this the new world order you are fighting for? If this the example of the society you are fighting to create, my motherland is better served by my King than your democracy.”

And is pulled out of the room as soon as he finishes his rhetoric. The leader stands at the now empty hall, except for him and the servant now curled in a corner, stunned at what has just happened. He turns towards the man in the corner, goes and hugs him uttering “We are all one, we are all equal..” and the scene fades away.

The interpretation

No other song, no other writing, no other painting, no other teaching, and no other anything has made me understand the meaning of democracy as this scene from an unknown movie. It essentially captures what every true believer in democracy fears, “autocracy”. Although none of the governments in the world would accept it, each one them is autocratic with varying levels of autocratic influence. Things like occupy movement have tried their best to expose this wolf in sheep clothing, little has transpired to reality.

Having said that, I do not mean to take the stand of a socialist by condemning free economy and enterprises, nor do I support the capitalist propaganda of talking in terms of wealth. But I believe there exists enough moral ground between the two in order to explore and settle. And probably the correct interpretation of democracy lies somewhere in that space.