The current blog post is to give a recount about building my personal website. To start things off, I will talk about how the project got started and its genesis. Next, I will document when work occurred and the difficulties that were happening that slowed down work. Fixing the errors that occurred in the first code draft will be discussed. After deployment of the website, other errors occurred, and those are considered. To conclude the blog post, I will discuss current problems with the site and possible improvements to be made in the future.
The idea for the website has been knocking around in my head for a while, but it only started in the summer of 2019. During that summer, I decided that I should get a website going so that employers can go to it and get more information about me all in one place. At the start, it was only a website built with https://www.wix.com/.
Kim and Josh from the lake mentioned it before and said that it was straightforward to make, so I tried it. Building the website took less than half an hour, and buying the domain and redirecting traffic to the Wix servers was probably the most challenging part.
I always wanted to build the website from scratch, because I had not done that before (only edited database applications). It was not until the Christmas holidays that work started on the site that you are browsing now. During those holidays, I determined that Flask, Python, and MySQL would be the technology to build the website.
The first sketches of how the website would look and function were made along with the database design/sketch. The design and sketches are stored in the public GitHub at this link:https://github.com/PhysicsUofRAUI/publicPersonalWebsite. They were very rough and do not adequately portray the website as it is now in some ways. The database sketch, in particular, is not up to industry standard, because I need to brush up on how to make database sketches properly.
When the sketches were done, I then proceeded to make all the files that would be necessary to create the website. In those files, I wrote, in natural English, all the functions that will be needed and what they will do. The design of the functions to display the blog posts had changed drastically from what they were when they were first made. To conclude the Christmas Holidays, I wrote some code but was not able to finish the website.
From the Christmas Break to the Winter Break at my university, very little work was accomplished. Course work took up more time, and finishing the website was pushed to the back. It was not until the Winter Break that the code was finished.
By finished, I mean, it was complete but not yet working. The first time I ran it, the code failed with an error. After fixing that error, another occurred, and so on. During the Winter Break, I think I was only able to fix 2 or 3 of the many errors to come, and work debugging slowed to a crawl.
Slogging through the errors was going slow until I met up with my friend Pablo. He helped me with a few of the bugs, and after that, the debugging seemed to snowball. It was about a day after meeting with Pablo that I was able to get the code running in development mode. Even after it compiled and was running in development mode, it was not yet ready to be deployed.
The main issues that were preventing deployment were database operations. When a blog post is added, the user can select a category, and I want those categories to be found from querying the database. A runtime error occurred because of that since the database operation was outside the view, and that caused issues with Flask’s context. The code below is what fixed the problem.
#
# The following functions return a query object but do not actually do
# the query. It is my understanding that the database is queried in the
# view that calls the form involved
#
def category_choices() :
return PostCategory.query
def sub_category_choices() :
return PostSubCategory.query
#
# The following is the form that will be calling the previous functions
#
class PostForm(FlaskForm):
"""
Form that lets the user add a new post
"""
title = StringField('Title', validators=[DataRequired()])
# here is where the function is called and listed as the query
# factory, and through some stuff within in Flask that I do not
# fully understand the database query is done in the view and the
# categories are listed in my form.
category = QuerySelectField('Category',
validators=[DataRequired()],
query_factory=category_choices)
sub_category = QuerySelectField('SubCategory',
validators=[DataRequired()],
query_factory=sub_category_choices)
content = TextAreaField('Content', validators=[DataRequired()])
submit = SubmitField('Submit')
There were also issues with the views not adding the posts properly and some other runtime errors. Most of them were minor, and lots were copy and paste errors that were easily fixed.
When all the errors were fixed, and the site was operating as it should in development mode, it was deployed. For deployment, I used https://www.digitalocean.com/, Nginx, and uwsgi. To accomplish the deployment, I followed the tutorial located at the following link: https://www.digitalocean.com/community/tutorials/how-to-serve-flask-applications-with-uswgi-and-nginx-on-ubuntu-18-04.
Now the first error that occurred very frequently was the dreaded ‘MySQL has gone away error’. At first, I thought it was only that the website is a very low traffic, which it is, but if that were all it was changing, the timeout values would have fixed that, and it did not. It turned out that my database sessions were not closing.
To close the session, I added a teardown function, which is shown below, but that still did not work. Eventually what I ended up doing is adding a .close(), .remove(), and .rollback() after each database call, an example of which is also shown below. After adding the teardown and all those functions, the errors went away. I am not sure if the teardown function needs to be there, but I am leaving it till I do more research.
#
# This is the teardown function and from what I understand it should be called
# everytime a database session ends but that does not seem to be its
# behaviour.
#
@app.teardown_appcontext
def shutdown_session(exception=None):
db_session.remove()
if exception and db_session.is_active:
db_session.rollback()
How the database operations worked:
#
# Because the teardown function is not have the intended effect all of my database queries look
# like this. The last three lines close the session out properly so the MySQL error does not
# occur.
#
blogs = db_session.query(Post).filter(Post.subcategory_id==subcategory).order_by(Post.id.desc()).offset(page * 5).limit(5)
more = db_session.query(Post).filter(Post.subcategory_id==subcategory).order_by(Post.id.desc()).offset((page + 1) * 5).first()
db_session.close()
db_session.remove()
db_session.rollback()
The next problem that had to be dealt with was the fact I was in the process of transferring all my travel blog posts, and that was a lot to load in one go. The obvious answer was to implement some pagination. Flask-SQLalchemy has a pagination function already implemented, but because I have my database set up using just SQLalchemy, the Flask-SQLalchemy pagination function did not work. It was not too difficult to implement a similar behaviour using the SQLalchemy functions, and that is shown below.
#
# The following was added to each view that has pagination.
# The variables prev_url and next_url are set and then past to
# the template. The template then implements pagination just
# as if I was using the pagination supplied by Flask.
#
if page != 0 :
prev_url = url_for('blogs.subcategory_blogs',
subcategory=subcategory, page=page - 1)
else :
prev_url = None
if not more == None :
next_url = url_for('blogs.subcategory_blogs',
subcategory=subcategory, page=page + 1)
else :
next_url = None
After these problems were fixed, I decided to declare that the first version of my website to be done. That does not mean there are no improvements I want to make to it, but it does mean that any improvements will have to wait for progress on other projects to be completed.
The biggest problem that is occurring right now is ‘CSRF Token is expired’. When I log out, clear the cache and try to log back in, I get ‘CSRF Token is missing.’ and it will not let me log in, and therefore edit the database until I go to the server and restart the application. I assume that the session which handles my login times-out, but I do not know why logging out and clearing the cache would not work. Overall it is more of a nuisance for me than anything else, so I am leaving it till later.
Two feature additions are on my wish list. One is to have an image file upload instead of what I have now, which is that I add the image to the static folder, and then I tell the database where the file is. It would be much more helpful to be able to load the image directly from my computer using the add photo function.
The second feature addition is adding a new main blog post category. That category would be Books, and it would contain reviews of books that I have recently read. It may also have ‘learning reviews’ from books where I give a chapter by chapter review of what I learned from the book. The ‘learning reviews’ would be books like ProGit, which I am currently reading.
None of these coding problems are likely to be solved soon, but the blog will be getting a lot of content added to it. For the moment, I am going to focus on transferring the rest of my travel blogs. The blogs from my travels while working in Belgium need to be added. There are also a few that I have not yet written, which I would like to write as well. When the Belgium blog is complete, all the posts transferred, and the last posts are written, I will work on improving my old blog posts.
When I travelled to Brazil and Morocco, my writing was not very good, and so I want to improve the posts I wrote from those times. I will be going one by one and adding content, fixing grammar, and hopefully improving the value of those early blogs.
That concludes my review of where the project that is my personal website is currently. The project will never truly be done until I take the site down, but for now, the work being done on it will be limited to the content creation and improvement of the Travel Blog.
P.S. For those who want to learn more about the code I will be doing a code walkthrough Sunday May 3rd, 2020 and it will be streamed live at the following link: My Code Walkthrough
The goal of this project is to have an excellent personal website so that I can easily share stuff related to my life in one place. Stories about my travel around Belgium (and the rest of Europe), Morocco, and Brazil will be on it so that I can relive it later when I am old. Personal projects of mine will also be documented so that others can see what I work on in my spare time and for me to keep track of it as well. Currently, the website is on Wix.com, which, if you are reading this, you know, however, this post is mostly about the version I am building from scratch.
I wanted to build this website primarily to bring together all the different blogs, code repositories and other places I keep stuff together. At the moment, I have three different blogs that are about my travel on three different exchanges:
Excuse the poor writing in the Brazil blog. I was much younger then and was very bad at written communication.
On top of those, I have been keeping code on my GitHub: https://github.com/PhysicsUofRAUI. I have also been documenting one of my projects on Hackaday: https://hackaday.io/physicsUofRAUI. That is in total five different places where I post or share stuff about my life, and I think it would be better to have all those things in one place.
I also have a lot of photos in my google photos. Most of those photos are shared on social media. However, I would like to have a more personalized way to look at them. I plan to add photos with a caption and mark them by category. It will then be easier to search through them and remember what was happening in the picture.
As previously stated, the website is on Wix.com for the time being, but a version built from scratch is under construction. The python microframework Flask is being used to construct the ‘from scratch site’. A templating language called Jinja is being used to generate the majority of the HTML. Not much front development has been thought of yet, but that will change.
Flask was chosen because I am familiar with it, and because it is flexible and powerful. I am familiar with Flask because I worked with it when working with Agriculture and Agri-Food Canada in Morden, making web applications to better view data that the bioinformatics crew was working with. I do not have any real empirical evidence to say that it is powerful and flexible, so I’ll go with because I think it is, and so do others.
A first version has been drafted and is currently being debugged. The debugging may take a while because I still need to learn to slow down and plan programs better. When the debugging is done, I will clean this up a bit and make it more presentable and then it will likely be deployed replacing the Wix.com version. I am hoping for it to be done by the end of the month. However, a more probable deployment date would be somewhere during the February break (week of the 17th).
This is a repost from my the former Wix website. The website is the site you are on now!