## Plotting functions with LaTeX & Tikz

I’ve been working on some notes for my students on Neural Networks. I am using Pandoc to write the text, and convert it back to LaTeX for PDF generation, and to HTML, for making the documents available online. In a future post I might talk about Pandoc, but for now, I want to share something I found on TeX Stack Exchange to plot functions. To make it clear how to use, I am presenting four simple examples: four common activation functions used in Neural Networks.

To start with, the well known sigmoid function: $a = \frac{1}{1+e^{-z}}$:

This can be achieved with the following tikzpicture environment:

\begin{tikzpicture}
\begin{axis}[
axis lines=middle,
xmax=10,
xmin=-10,
ymin=-0.05,
ymax=1.05,
xlabel={$x$},
ylabel={$y$}
]
thick, blue] {1/(1+exp(-x)};
\end{axis}
\end{tikzpicture}

Now, for the tanh, defined as $a=\frac{e^z – e^{-z}}{e^z + e^{-z}}$, you get:

With the following code:

\begin{tikzpicture}
\begin{axis}[
axis lines=middle,
xmax=10,
xmin=-10,
ymin=-1.05,
ymax=1.05,
xlabel={$x$},
ylabel={$y$}]
thick, blue] {(exp(x) - exp(-x))/(exp(x) + exp(-x))};
\end{axis}
\end{tikzpicture}

I was already loving it, but it was awesome when I tested the ReLU math, and it worked as expected: $a=\max(0, z)$

This one can be obtained using the simple…

\begin{tikzpicture}
\begin{axis}[
axis lines=middle,
xmax=6,
xmin=-6,
ymin=-0.05,
ymax=5.05,
xlabel={$x$},
ylabel={$y$}]
\addplot [domain=-5.5:5.5, samples=100, thick, blue] {max(0, x)};
\end{axis}
\end{tikzpicture}


And now, although not being extremely interesting, the Leaky ReLU (well, not implemented exactly as the definition, just to get it a little more easy to read): $a=max(0.001 \times z, z)$

With the cheating…

\begin{tikzpicture}
\begin{axis}[
axis lines=middle,
xmax=6,
xmin=-6,
ymin=-1.05,
ymax=5.05,
xlabel={$x$},
ylabel={$y$}]
thick, blue] {max(0.1 * x, x)};
\end{axis}
\end{tikzpicture}


And to conclude, and just for reference, do not forget to load the required packages:

\usepackage{tikz}
\usepackage{pgfplots}

## IGI Global: the clown of scientific publishing?

I am not sure how I agreed to write a chapter for a book to be published in IGI Global. Probably, being edited by a friend that invited me personally to send a proposal made the difference.

I have my contribution ready, but starting to think on just forgetting it. Why? Because IGI Global is, surely, kidding with me. They have a set of rules for their contributions, and somewhere in the middle, they say, and I quote:

LaTex. LaTex files are NOT accepted because they are not compatible with IGI Global’s typesetting program. As an alternative, we require that you use MathType (see “Equations” below).

First, dear IGI, when not possible to use the fancy form of LaTeX, the latest X should be in uppercase. Second, if hey are not compatible with your typesetting program, that is probably because you are using the wrong typesetting program. And, no, LaTeX is not useful only for math. Please learn what LaTeX is, try to use it, then evaluate how it can be useful or not for your editorial requirements.

Third (or fourth, I think I will stop counting), look to other publishing houses. Who are your adversaries? Springer, probably. Do you know they use LaTeX? Yeah, they do! And they create good quality document. Of course they do, they use LaTeX. And no, I have an IGI book, and no, your books does not have typesetting quality. I am sorry.

Finally, because I have some hours to lose formatting the chapter, if you want us to use Microsoft Word, please create a template in Word. Do you know what that is? You know how it can be useful? Do you? I am sure you don’t.

## Atom.io

About two months ago I discovered the existence of Atom.io, a free editor from the GitHub team. This fact, being from the GitHub team, made me wonder. There are not good editors for Windows. I usually use Notepad++ that is great, but somewhat limited on extensibility. When I use Mac OS X or Linux, I usually use Emacs or Vim. I know there are versions of Emacs or Vim for Windows, but I got curious on the Atom.io editor. But unfortunately I did not like it. It is quite heavy starting, the UI is not very polished and there isn’t native support for LaTeX. I know the idea of this editor is to be extensible, but when something gets completely community driven, diverse modules or packages for the same purpose appear, and it gets difficult to know what to install.

Nevertheless, I did not quit at first. I got into the package manager and tried to install a LaTeX package. The first package I found should, at least, allow me to compile a tex file directly from the editor. After about 3 minutes installing a package (too much time for such a limited package) I tried to compile the LaTeX document and I did not get any feedback. Neither that the plugin was invoked or that it wasn’t; if the file was compiled, or not. Nothing. I didn’t even get LaTeX highlight.

Please, atom.io crew. Look into Notepad++. Look to its size. Look to how fast it starts. Look to the amount of different languages syntax he supports.

So, at the moment, wondering if I’ll install Sublime, Gvim or some Emacs.

## XeTeX

You already knew I am a LaTeX fan. I finally got the courage to go and experiment XeTeX. In fact, I was almost forced, as I am the Portuguese official translator for The Not So Short Introduction to LaTeX. By the way, you can buy the Original (English) or the Portuguese versions from lulu.com.

OK, enough advertisement. XeTeX is one of two Unicode TeX motors. The other is named luaTeX and I think (didn’t have the time to look into it yet) and is scriptable in the lua programming language. What does it mean to be an Unicode engine? It means it expects your text to be written in Unicode (in fact, UTF-8). This gives you the ability to typeset any symbol or character you like, as far as it is available in the Unicode tables, and you have a font that is able to render that character.

This leads to a small problem. In fact, a problem that the main alternative (Microsoft Word) also have. You need to have the fonts installed in your system (or in the folder where your tex document is, or any other place where tex would search for include files). But this gives the ability to select any font, and any font for any language. I was able to typeset about 30 translations for “Good Morning”, including Arabic, Persian, Korean, Japanese, Traditional Chinese, Georgian, Greek, Hebrew, Urdu and others. Note that some of these languages are written right to left, other left to right. But as far as you are able to include the characters in your Unicode file, then with the correct fonts and some TeX packages, you will be able to typeset them.

If you want to give a try, look into fontspec and polyglossia packages for a start. Who knows if in the future I do not add another post with further details on how to use XeTeX.

## Text::BibTeX 0.40 Released

Normally I do not post on every perl module I release. And that is good, or you will be feeling spammed.

Why this module is different?

Because I adopted Text::BibTeX a long time ago, and had a lot of complaints about its installation mechanism. This was mainly due to the fact that Text::BibTeX depended on a C library that needed to be installed prior to the perl module. The C library installation was easy on generic Unix platforms but was a pain to compile under Windows.

After lot of work I managed to include the library C code in the Perl module (now Text::BibTeX has no dependencies on the library), and managed to include code to compile the library in Windows, using the Strawberry Perl distribution (that includes a mingw C compiler).

The package is needing heavy tests, but it seems usable for most users. Probably I will post on the details about its build system in a later post.

## LaTeX discussion list (in Portuguese)

Perl-hackers is now hosting a LaTeX mailing list for help and discussion on LaTeX. You can subscribe sending an email to latex-request@perl-hackers with ‘subscribe’ in the subject.