## Plotting functions with LaTeX & Tikz

I’ve been working on some notes for my students on Neural Networks. I am using Pandoc to write the text, and convert it back to LaTeX for PDF generation, and to HTML, for making the documents available online. In a future post I might talk about Pandoc, but for now, I want to share something I found on TeX Stack Exchange to plot functions. To make it clear how to use, I am presenting four simple examples: four common activation functions used in Neural Networks.

To start with, the well known sigmoid function: $a = \frac{1}{1+e^{-z}}$:

This can be achieved with the following tikzpicture environment:

\begin{tikzpicture}
\begin{axis}[
axis lines=middle,
xmax=10,
xmin=-10,
ymin=-0.05,
ymax=1.05,
xlabel={$x$},
ylabel={$y$}
]
thick, blue] {1/(1+exp(-x)};
\end{axis}
\end{tikzpicture}

Now, for the tanh, defined as $a=\frac{e^z – e^{-z}}{e^z + e^{-z}}$, you get:

With the following code:

\begin{tikzpicture}
\begin{axis}[
axis lines=middle,
xmax=10,
xmin=-10,
ymin=-1.05,
ymax=1.05,
xlabel={$x$},
ylabel={$y$}]
thick, blue] {(exp(x) - exp(-x))/(exp(x) + exp(-x))};
\end{axis}
\end{tikzpicture}

I was already loving it, but it was awesome when I tested the ReLU math, and it worked as expected: $a=\max(0, z)$

This one can be obtained using the simple…

\begin{tikzpicture}
\begin{axis}[
axis lines=middle,
xmax=6,
xmin=-6,
ymin=-0.05,
ymax=5.05,
xlabel={$x$},
ylabel={$y$}]
\addplot [domain=-5.5:5.5, samples=100, thick, blue] {max(0, x)};
\end{axis}
\end{tikzpicture}


And now, although not being extremely interesting, the Leaky ReLU (well, not implemented exactly as the definition, just to get it a little more easy to read): $a=max(0.001 \times z, z)$

With the cheating…

\begin{tikzpicture}
\begin{axis}[
axis lines=middle,
xmax=6,
xmin=-6,
ymin=-1.05,
ymax=5.05,
xlabel={$x$},
ylabel={$y$}]
thick, blue] {max(0.1 * x, x)};
\end{axis}
\end{tikzpicture}


And to conclude, and just for reference, do not forget to load the required packages:

\usepackage{tikz}
\usepackage{pgfplots}

## IGI Global: the clown of scientific publishing?

I am not sure how I agreed to write a chapter for a book to be published in IGI Global. Probably, being edited by a friend that invited me personally to send a proposal made the difference.

I have my contribution ready, but starting to think on just forgetting it. Why? Because IGI Global is, surely, kidding with me. They have a set of rules for their contributions, and somewhere in the middle, they say, and I quote:

LaTex. LaTex files are NOT accepted because they are not compatible with IGI Global’s typesetting program. As an alternative, we require that you use MathType (see “Equations” below).

First, dear IGI, when not possible to use the fancy form of LaTeX, the latest X should be in uppercase. Second, if hey are not compatible with your typesetting program, that is probably because you are using the wrong typesetting program. And, no, LaTeX is not useful only for math. Please learn what LaTeX is, try to use it, then evaluate how it can be useful or not for your editorial requirements.

Third (or fourth, I think I will stop counting), look to other publishing houses. Who are your adversaries? Springer, probably. Do you know they use LaTeX? Yeah, they do! And they create good quality document. Of course they do, they use LaTeX. And no, I have an IGI book, and no, your books does not have typesetting quality. I am sorry.

Finally, because I have some hours to lose formatting the chapter, if you want us to use Microsoft Word, please create a template in Word. Do you know what that is? You know how it can be useful? Do you? I am sure you don’t.

## Any.Do – The best task tracking system ATM

For a long time I have been trying different ways to track my stack of tasks. I tried different approaches, from using paper (it is fun!), Calendar Mac App, Wunderlist, and recently Any.Do.

Paper is probably the best, as you can write freely, draw, scratch, rip, and other physical activities that are quite fun. But, unless you get a small notebook, it gets hard to be with you all the time. You could also scratch on a smart phone device, like Samsung S Note app. But when I am working in my laptop, I do not like to have to take the phone to add or remove entries. So, next natural step is to use a tool that syncs tasks between devices.

Wunderlist is cool. Cool enough for me to use it for some time. It has the fun fact that it keeps finished tasks scratched, just like you might do in paper. Nevertheless, the app is (or was, when I tried it) quite limited. You could have different todo lists, but it was hard to see them all at once. You could add some manual tags to tasks (I did that for some time), but it got boring. And it wasn’t easy to sort tasks. You needed to, somehow, change its date.

I tried for some time Evernote for that. It allows to add tick boxes to your notes. But again, not easy to see all tasks in one screen, unless you add them all in the same sheet. And if you do, it gets complicated to manage the order, and the categories, with lots of cut, copy and paste.

So, recently I am using Any.DO. It is also available for Mac, Web, and Android, so easy to sync around. And it has three main ways to see your tickets: per category, per time, or per importance. And in each of these three views, you can sort easily your tasks just by moving them around. I am very satisfied (for now, at least) with it. But I still wonder how much more fun and efficient it might get with the paid version. But a monthly (or yearly) subscription is not exactly what I want right now. Nevertheless, if I could buy the app (and not a service) I might be buying it soon.

## Atom.io

About two months ago I discovered the existence of Atom.io, a free editor from the GitHub team. This fact, being from the GitHub team, made me wonder. There are not good editors for Windows. I usually use Notepad++ that is great, but somewhat limited on extensibility. When I use Mac OS X or Linux, I usually use Emacs or Vim. I know there are versions of Emacs or Vim for Windows, but I got curious on the Atom.io editor. But unfortunately I did not like it. It is quite heavy starting, the UI is not very polished and there isn’t native support for LaTeX. I know the idea of this editor is to be extensible, but when something gets completely community driven, diverse modules or packages for the same purpose appear, and it gets difficult to know what to install.

Nevertheless, I did not quit at first. I got into the package manager and tried to install a LaTeX package. The first package I found should, at least, allow me to compile a tex file directly from the editor. After about 3 minutes installing a package (too much time for such a limited package) I tried to compile the LaTeX document and I did not get any feedback. Neither that the plugin was invoked or that it wasn’t; if the file was compiled, or not. Nothing. I didn’t even get LaTeX highlight.

Please, atom.io crew. Look into Notepad++. Look to its size. Look to how fast it starts. Look to the amount of different languages syntax he supports.

So, at the moment, wondering if I’ll install Sublime, Gvim or some Emacs.

## Octave Wonders

Was looking around the Octave language… and requested help about the fact function.

Display an amazing and random fact about the world’s greatest hacker.

I decided to experiment:

Richard Stallman once used GDB to reverse-engineer Windows 7 into a free operating system – able to run on GNU Emacs!

And tried again

Richard Stallman won a Suduku that started with only one number in each line

And now I wonder, I will I be able to work again?

## Panoramic Photos – Hugin

Hugin is the only free alternative to AutoPano Giga/Pro that I have found. In these cases the number of photos is smaller (and mostly horizontal). This tool is not as good as AutoPano in guessing the position of photos, but does a good overall job.

## Panoramic Photos – AutoPano Giga

These are my two first experiences with AutoPano Giga. They have plenty of problems. One of the photos has a lot of inconsistencies, most of which are my fault, for moving while taking the photos (and not just rotating over an axis). Nevertheless, I like these pictures.

## XeTeX

You already knew I am a LaTeX fan. I finally got the courage to go and experiment XeTeX. In fact, I was almost forced, as I am the Portuguese official translator for The Not So Short Introduction to LaTeX. By the way, you can buy the Original (English) or the Portuguese versions from lulu.com.

OK, enough advertisement. XeTeX is one of two Unicode TeX motors. The other is named luaTeX and I think (didn’t have the time to look into it yet) and is scriptable in the lua programming language. What does it mean to be an Unicode engine? It means it expects your text to be written in Unicode (in fact, UTF-8). This gives you the ability to typeset any symbol or character you like, as far as it is available in the Unicode tables, and you have a font that is able to render that character.

This leads to a small problem. In fact, a problem that the main alternative (Microsoft Word) also have. You need to have the fonts installed in your system (or in the folder where your tex document is, or any other place where tex would search for include files). But this gives the ability to select any font, and any font for any language. I was able to typeset about 30 translations for “Good Morning”, including Arabic, Persian, Korean, Japanese, Traditional Chinese, Georgian, Greek, Hebrew, Urdu and others. Note that some of these languages are written right to left, other left to right. But as far as you are able to include the characters in your Unicode file, then with the correct fonts and some TeX packages, you will be able to typeset them.

If you want to give a try, look into fontspec and polyglossia packages for a start. Who knows if in the future I do not add another post with further details on how to use XeTeX.

## Unity 3D

In the last days I’ve been using Unity3D. Unity3D is a electronic games engine. It has been developed by some years, but in the last years a free version, with some limited features, was made available.

Now, why am I, somebody who likes to write real code, playing with this kind of software, where most of the work is done by the mouse?

Sometimes we have to do things that we do not like, or we aren’t ready to. And in this case, I was chosen to teach a course on Applied Game Development, and IPCA, in the second year of the graduation students of a Game Engineering Degree.

What can I say on Unity3D? Well, first of all it is an IDE merged with a graphical editor, where you can prepare your scenes. Then, you can script your scenes with three different languages, C#, JavaScript or Boo, a Python dialect. In fact, JavaScript isn’t JavaScript (it is a JavaScript dialect) and I wonder whether C# is really C#. Any way, you can script different parts of your game in any language. This is just great.

Then, your code will run in Mono. Yes, Mono is the OSS .net implementation, that was started by Miguel de Icaza (as far as I can remember). I am sincere, never though it would be being used as it is in so few years.

One of the great things on Unity is the ability to, with some plugins, export (or build) your game for a variety of platforms, ranging from the top console market (Wii, XBox, PS3) to the top mobile market (iOS and Android) passing by the major PC architectures/operating systems, and touching the Web.

At the present moment I am having some fun with Unity. I am just sorry there isn’t any academic program where the students (and teachers) would be able to use the Pro version with a lower price (current Pro license is \$3000, too much for a public Portuguese academic institution).

## Calibre developer shows lack of interest

A friend lent me a Kindle DX. I installed Calibre to perform some simple conversion from .epub format to .mobi format. My first experiment was with the Open Dictionary for Portuguese (Dicionário-Aberto). This dictionary takes 5MB on ePub format. Let me remember you that ePub is a zipped format. It, after unzip, takes 25 MB. The dictionary is pure text.

The conversion from ePub to Mobi took about 45 minutes. That’s not a problem. But took more than 1.9 Gigabytes of disk space. Yes, you are reading the unities correctly. I found this value kind of absurd and though that, although not a relevant thing to fix right now, it would be interesting to investigate why it takes so much disk. By the way, the final Mobi file is around 15MB.

I’ve opened a ticket on Calibre, marked it a wishlist: find out why Calibre takes so much disk space performing a simple format conversion. I did not ask for it to be fixed. I did not complain. Just said it took that disk space, and added a pointer to the file that I used.

I was expecting the developer to be intrigued about this. But unfortunately he shown to be stupid. Sorry, I know this is a hard word, but it is the only explanation I have to this answer:

How much space it takes is how much space it takes. I have no interest in optimizing disk space usage.

Note that I wasn’t expecting a fix right away. An answer stating the there is a to-do list with more important things, would be OK. But closing the ticket with a wont fix made me delete the application from the disk. Would prefer to develop my own conversion tool than to know I would be using this guy software.