Errors Encountered Setting Up Django Project

I recently got a project, based in Django. Like all new project, I had to set up the development environment before working on it. The post is about issues I troubleshoot while setting up the development environment:

  • Python Virtual Environment: When a tool like virtualenv is setting up a virtual environment, it creates it base on the computer’s Python version unless we specify a different version in the command. The project was Python 2.7, so the setup encountered errors because my computer use Python 3 natively. If you are not sure what Python your computer is running on, a simple python –version will show it. To ensure there are no error, I use the command virtualenv -p python2.7 venv.
  • OS & Package Dependencies: Apparently Linux require manually installing several dependencies for matplotlib & scipy. At least according to the project README, which was written for Linux. As MacOS user, I didn’t have to worry about that.
  • Compiler in OS: I do, however, have to worry about command invocation not finding my compiler when I doing npm install. To resolve, use command CXX=clang++ npm install.
  • dyld: Library not loaded: executable_path/../.Python” => Homebrew & Python Path: The errors occurs after ./manage.py runserver, and it is a result of broken Python symlink after a Homebrew update I did for another project. Here are the steps to resolve:
    1. Deactivate the virtual environment.
    2. Upgrade the virtual environment tools. In my case, I did pip install –upgrade virtualenv and pip install –upgrade virtualenvwrapper.
    3. Delete broken symlink using find venv -type l -delete while inside the project directory.
    4. Now redo the virtual environment set up. In my case, it was virtualenv -p python2.7 venv => source venv/bin/activate => pip install -r requirements.txt.

Solve node-gyp compatibility issues in npm install

I had to reinstall a project built in Keystone.js – a JavaScript CMS – lately, and I encountered some compatibility issues that I encountered before. I totally forgot about it, and took me a while to realize it was the same issues I encountered before.

Mainly, the issue was node-gyp. Node-gyp is a command-line tool written in Node.js for compiling addon modules for Node.js. It bundles Chrome’s team’s gyp project, which helps with cross-platform issues. Alas, the tool itself ended up causing its own issues!

Node-gyp error
The error message

For me, there are two problem:

  1. It does not support Python 3.
  2. It needs make – a build automation tool – with its variable CXX set to proper C/C++ compiler toolchain.

In my case, my system’s default Python is 3 and the CXX is not always set to the right toolchain. So, first remove the local directory’s node_modules folder if you did a failed¬†npm install earlier, then input this in the command line:

CXX=c++ npm install –python=python2.7

Most node-gyp rebuild error should disappear!

KeystoneJS success
Dadah!!! Keystone.js now running!

What I Have Been Doing 2016-2017

Before I post my 2017 Resolution, let’s list out and organize out what I am doing now. Cause honestly, I think I am doing too much at once. I need to trim down and focus. Ok, so here goes!

  • MIT’s 6.00.1x Introduction to Computer Science and Programming Using Python course
    • Just started this week. I am regretting it a bit now, cause it is asking my to install Anaconda right away, and the last time I did that it messed up my localhost Django practice page – conda does not get along with pip and virtualenv. This is definitely going to be more focus on data science than just Python. I like Python a lot, so a lot of time I forget most people use it for data science, not web development.
  • Functional Programming, cause I want to learn React.
    • This is growing into a monster. I only know basic vanilla JS and jQuery from like 2 years ago when I first got into web dev. One month of study, and I am now on 3 different tutorial cause everyone talks about a different aspect of modern JS. This is what happened:
      1. At first, someone recommended me to watch funfunfunction’s Youtube channel Functional programming in JavaScript. Oh, he’s funny indeed… Oh my god how much had JavaScript changed in the last 2 years!?!?!? Promises? Currying? Map? Filter? What Da H?
      2. Google. Google. How come the functions are all written differently (<— was not fully aware of the whole ES5-ES6-ES7 tangle of mess)
      3. Maybe my foundation just isn’t good. I had never read the holy grail JS book, Eloquent JavaScript. They were just talking about how good it is in the Women Who Code JS Meetup. Let’s do it. Oh, Wes Bos have a really cool video called JavaScript 30? Even better. I will make myself program once a day in JS! Let’s do both.
      4. Combining Eloquent’s Chapter 5 content with Wes Bos’ Array Cardio part 1 and part 2 was really good in helping me to finally understand higher order functions like map() and reduce().
      5. Decided to do Eloquent exercise – which is in ES5 – with ES6/7 instead to learn the differences between the versions.
      6. Wait, wait, wait. Why is object and prototype completely different between ES5 and ES6!? And the constructor! The class! What is even count as object in JS? (<— comes from a C++/PHP7 background)
      7. Found Secrets of the JavaScript Ninja in Barnes & Nobles. Great, another book. But I need it, cause it will explains what on earth is going on with this function-object JavaScript mess.
    • Correction, I am on 1 Youtube channel, 1 video tutorial, and 2 books all at once.
    • Told ya it’s a monster.
  • Doing a side project website of Drupal cause I wanted to brush up on for my internship. Which resulted in learning Bootstrap on Treehouse, cause the theme I selected is based on bootstrap. At least this one’s almost done.
  • The Drupal site is worked on while reading Beginning Drupal 8 by Todd Tomlinson.
  • Revamp of my website, which is PHP.
  • Yea, I am taking on way too much. I think I am going to put a pause on my Drupal project and reading (currently chapter 10)… Should work on my web page first.

Estimating, Mobile, and Websocket on Django

Recently, I went to my first Django Meetup and it was totally worth the trip, even the part where I accidentally took a wrong turn and ended up in Treasure Island instead.

…..

What? The highway entrance to the bridge is literally next to the building! It actually wraps around it, which I thought was pretty cool, but anyway – tip to those going to Four Point: Don’t go up the ramp!

I like the inside too. The Meetup area where the group ate pizza is beneath an interior overlook. Some of the employees had paper decoration hanged up, and they range from pizza to Powerpuff Girls (They really like their pizza, don’t they?). There were rows of wooden benches to the right for the presentation, but for the first half hour, everyone is just enjoying the food.

Having finally started to learn Django and on my first web developer job hunt, it was pretty fun to hear from working Python developer at the table talk about how a Python project typically progress. For example, I got to hear about the typical requests from ecommerce clients and how there are features that most ecommerce clients eventually decides to get even if the clients didn’t thought of it at first. There were discussion of language version craziness (one had a client that use all version of Django starting from… 1.4 I think? Django is 1.10 right now, by the way). Of course, framework differences were also brought up, though being a Django meetup, everyone had found Django to be fairly likeable.

I wouldn’t go into detail about the presentations, since all notes and videos are available to the Meetup event site, Django + Ionic & REST Websockets API with Django Channels , but here are some thoughts:

The first talk was about a Python Estimator. With the name, at first I thought it have to do with finance or concurrency. Turned out it was more about improving the way data scientist shares data with back-end engineers.

The second talk have more to do with the presenter’s journey through building a well-functioning web app via Python and Ionic that tracks food truck – on his spare time. Talk about discipline!

Sam’s talk was about RESTful Socket. I had not read too much about the event profile until the day of the Meetup, so I was surprised to realize that Sam was an engineer at HouseCanary, the company that hosted the hackathon I participated. He was also one of the guys at the table I was sitting in. Realizing that makes me look even more forward to the presentation, since he sounded very knowledgeable at the table.

It is definitely the most technical talk of the day, leaving me much more educated and slightly awestruck about how much more there is to REST and websocket. I had used a bit of REST APIs before, but while I have read some definitely of both of REST and websocket, the definition in relation to their application plus their differences have never really sunk in until this talk. Though I have yet to learn channel, signal, and websocket, the talks today left me eager to learn more.

Deploying my Python Slackbot onto Heroku

I just build a Slack chatbot in Python within a night.

With all the hypes about building a chatbot, that was way easier than I imagine.

I pretty much just followed the tutorial at Full Stack Python: How to Build Your First Slack Bot with Python, which was pretty quick and easy. I hilariously tried to test out my slackbot for 10 minute before realizing I was in localhost. Duh.

So here is the long story version of my troubleshooting:
Most of the time I took to build the slackbot was actually the deployment to Heroku, since I have only ever used it once. When the warning “No default language could be detected for this app. HINT: This occurs when Heroku cannot detect the buildpack to use for this application automatically.” popped up, I was pretty clueless (“What’s a Buildpack?”).

So, I went to the Heroku site and added the Python Buildpack. Since it was giving out error message about not detecting Python, it reminded me that I was working with a virtual environment. Thus, I should do a pip freeze > requirement.txt so the deployment environment knows what to load. So I did that. Let’s git push again…

Python app detected Warning: Your application is missing a Procfile. This file tells Heroku how to run your application… Installing python-2.7.12

Python app is detected, but missing a Procfile? Isn’t that just for Django (my only encounter with Procfile so far is the Django Girls tutorial)? Also, why is it installing Python 2.7.12?

A quick Google revealed that I can create a runtime.txt file to state which Python version I want. I also did some reading on Procfile in term of Heroku and Python deployment. For apps, the example provided was web: gunicorn gettingstarted.wsgi and web: gunicorn hello:app. The later seemed to be more for frameworks like Django, so I went with the later and added web: gunicorn starterbot.wsgi –log-file –. I then pip install gunicorn and redo pip freeze > requirement.txt.

Along the way, I also saw another Heroku deployment tutorial that reminded me that I should use .gitignore. That’s right, I shouldn’t deploy my virtual environment files for security reason. That’s why the “SLACK_BOT_TOKEN” and “BOT_ID” are exported in the first place! So I went and delete my Heroku app, deleted my .git, create the .gitignore and Procfile file, and deploy my app again.

The slackbot, however, didn’t seemed to be running. In the slackchat, the dot that indicate activity status remained grey instead of green. Doing heroku logs revealed that “ImportError: No module named ‘starterbot.wsgi’; ‘starterbot’ is not a package“. So its the Procfile. After some more reading and I narrowed down to the Heroku statement that Procfile “…explicitly declares what command should be executed to start your app.“. So, I tried web: python startbot.py, because that’s what get the program running, right?

That does get it running! At first, it seems to work well, but the slackbot keeps shutting itself down after a while, and it never respond to my command. Another heroku logs reveals that “Stopping all processes with SIGTERM… Process exited with status 143 …Error R10 (Boot timeout) -> Web process failed to bind to $PORT within 60 seconds of launch… Stopping process with SIGKILL… Process exited with status 137… State changed from starting to crashed“.

It seemed something was wrong with the PORT variable. So, I went to the Heroku site and manually added the PORT config variable, testing the deployment with different port. None seemed to work.

I did a search of “heroku chatbot port”. Surprising, I found the answer in a Node.js tutorial – the author,¬†Luciano Mammino, used “worker: node bin/bot.js” for his Procfile. What would happen if I use “worker: python starterbot.py“?

That got it running, permanently, without any error message about port binding!

My bot is still not outputting anything when I was typing commands though. I added in error checking statement in the if-else statement and eventually realized I have missed a simple colon sign in my command (“@starterbot:”). But once I started adding that, my slackbot worked!

The short, listed version:

  1. heroku create
  2. Find the corresponding app in the Heroku account. Go to Settings and add “SLACK_BOT_TOKEN” and “BOT_ID” to the Config Vars.
  3. pip freeze > requirements.txt
  4. echo “python-3.5.2” > runtime.txt (or whatever Python version you are working with)
  5. echo “worker: python starterbot.py” > Procfile
  6. Add the virtual environment directory name and *.pyc into .gitignore.
  7. Do git add, commit, and push.The bot should be up and running!