UP | HOME

Tips and tricks: the non-geek scientist

Table of Contents

Emacs

Needless to say how powerful emacs is. However, it can quickly become criptic. The Practical Emacs Tutorial by Xah Lee is an excellent reference for those like me who already know something, but very little.

Miscellaneous

  • hidden files: .class are not showing with TAB-completion; customize variable completion-ignored-extensions

Make programming easy for a non-geek scientist

There are many tools to develop complex codes, so-called IDEs (Integrated Development Environment). However, I tend to try and use as little CPU and power resources as possible, and therefore, try to minimize the number of softwares. Actually, developping with emacs is more than sufficient: it is extremely efficient. I have been developping codes with more than 1e5 lines with emacs. However, I am not a professional code developper and therefore, the following may only be of interest for scientists.

Latex code folding via outline minor mode

Adding the following lines to my .emacs.d/init.el allows to circulate through the Latex file. This is based on this reference.

(add-hook 'prog-mode-hook 'outline-minor-mode)
(global-set-key [M-left] 'outline-hide-body)
(global-set-key [M-right] 'outline-show-all)
(global-set-key [M-up] 'outline-previous-heading)
(global-set-key [M-down] 'outline-next-heading)
(global-set-key [M-S-left] 'outline-hide-subtree)
(global-set-key [M-S-right] 'outline-show-subtree)

Fortran code

Aim: have something as simple as org-mode to circulate within a program (e.g. Fortran). After spending some time, and thanks to brilliant emacs users, I found a minimal, though, sufficient, setting that does not need any additional package. This applies to Fortran, Latex codes, etc.

In my .emacs.d/init.el, I added the following:

;; this is from: https://www.reddit.com/r/emacs/comments/e2u5n9/code_folding_with_outlineminormode/
(add-hook 'prog-mode-hook 'outline-minor-mode)
(defun my-outline-fortran ()
"Fold definitions in Fortran."
(setq outline-regexp
(rx (or
;; Module and interface blocks.
(group (group (* space)) (or "module" "interface"))
;; Procedures and type definitions.
(group
(group (* space))
(*? (or "pure " "impure " "elemental "))
(or "function" "subroutine" "interface" "type" "type,")
(group (+ space)))))))
(add-hook 'f90-mode-hook 'my-outline-fortran)
;; The following is needed if you did not define these keys somewhere else.
;; (global-set-key [M-left] outline-hide-body)
;; (global-set-key [M-right] 'outline-show-all)
;; (global-set-key [M-up] 'outline-previous-heading)
;; (global-set-key [M-down] 'outline-next-heading)
;; (global-set-key [M-S-left] 'outline-hide-subtree)
;; (global-set-key [M-S-right] 'outline-show-subtree)

Editing

  • show line number on the left in the window: M-x linum-mode
  • Disable fontification in math mode M-x customize-variable font-latex-fontify-script <RET>
  • Scrolling "a la EXCEL" with a top row that is always visible: the trick is to split into two buffers, with the upper one containing only one row: C-u 1 C-x 2
  • Replace carriage return with a blank

    M-x replace-string C-q C-j ENTER space # C-q is the escape character and C-j is for the newline
    

Faces

  • Check the face of the character below the cursor:
    • M-x describe-face: the easiest
    • C-u C-x = will give many information and a link towards the specific face. If you are in a console or are doing emacs -nw in a terminal, instead of clicking, put the cursor on the appropriate element and press RET.
  • Inspect the value of frame-background-mode and customize it if needed

Emacs/org-mode

Using the truely fantastic org-mode tool makes emacs the most powerful tool for my everyday life as a scientist and teacher. The org-mode saves you a huge amount of time when producing rich READMEs, web pages

Editing

  • Commenting: C-c ; to comment an entire entry at once
  • Narrowing: to isolate an entry, use C-x n s and C-x n w to get back to the full TREE.
  • Navigation
    • C-c n/p: Next/Previous heading, regardless of level
    • C-c f/b: Next/Previous heading, same level
    • C-c u: to go backward to upper level
  • To display all links in plain text: M-x org-toggle-link-display
    • if you want to keep this behavior permanently, simply add (setq org-descriptive-links nil) to your init file

Tables

  • Build a table from a tabular text: copy/paste the table, then select it with C-x C-x and then C-c |
  • Operating on columns
    • swap two columns: ALT + left/right
    • remove current column: ALT + SHIFT + left/right
    • insert column to the right: ALT + SHIFT + right
  • Duplication
    • SHIFT + ENTER duplicates the current field in the row below; overwrites the content of the field;
    • Similarly for rows, with up and down
  • Clear cell content: C-c SPC
  • C-c - insert a ----- line below the cursor
  • #+TBLFM to perform operations on columns of a table; several ones can be used, one per line, to be placed right below the table; C-c C-c applies the formula; output format controlled as follows: #+TBLFM: $2=9*$1**-2.3;%4.3g.
  • To add a row counter: #+TBLFM: $1=@#. This would start from row 1. Can be adjusted.
  • Sorting: put the cursor in the column to be sorted, and type C-c ^ and chose a value from the menu in the bottom menu bar.
  • C-c } Toggle the display of row and column numbers for a table, using overlays.
  • C-c + will compute the sum of the column above.
  • @>$3=vsum(@2..@-1)
    • @> means 'last row'
    • $3 is the 3rd column
    • vsum is a vector sum; may be used either w/ columns or rows
    • @-1 means last row above the current location, hence penultimate;
  • See here or here (orgmode.org) for more, and even more.

Appearance

  • org-indent-mode: see here

Links

  • C-c C-l to edit the link
  • M-x org-toggle-link-display to stwitch between collapsed/extended view of links

Using org-agenda

  • General link here
  • C-c a to open org-agenda dialog box

Using code blocks

  • To prevent org-mode and babel to evaluate code blocks during export, put this at the top of the org document:

    #+PROPERTY: header-args :eval never-export
    

Exporting

  • To export: C-c C-e and then follow the menu
  • HTML export
    • C-c C-e P f will export the file within a project
    • INFOJS_OPT must be given once and for all and on a single line
    • #+OPTIONS: num:t will disable the folding property of infojs
  • LATEX export: typical sequence to produce a PDF is C-c C-e l p

Oral presentations

org-mode can make it easy to produce oral presentation documents. I suggest to explore two ways: one is the reveal.js export which produces HTML presentations; nice, but quickly criptic. I personally decided not to go ahead with reveal.js essentially because it was quickly becoming lengthy and resembling latex files with a significant amount of useless characters.

So I moved to using the foiltex package which will sound completely stone-age to those who knew it from the 2000s, and probably to anyone indeed; yet, this produces simple and focused, high quality, PDF documents. And, in combination with org-mode, I finally have a way to produce latex-compatible slides with highly efficient files.

There must be some way to improve on what I did which started from this reference.

  • An example of a PDF file produced with this method may be seen here.
  • The following lines are adapted from this reference. I added them to my ~/.emacs.d/myinit.org.
;; Foiltex with emacs org-mode
;; from https://github.com/tomfaulkenberry/orgFoils
(require 'ox-latex)
(add-to-list 'org-latex-classes
'("foils"
"\\documentclass{foils}"
("\\foilhead{\\color{MidnightBlue}\\bf\\huge\\raggedright\\rule{10pt}{3ex}\\hspace{1em} %s}" . "\\section{%s}")
("\\foilhead{\\color{RedOrange}\\Large\\raggedright\\rule{10pt}{2ex}\\hspace{1em} %s}" . "\\subsection{%s}")
("\\foilhead{\\color{Blue}\\large\\raggedright\\rule{1ex}{1ex}\\hspace{1em} %s}" . "\\subsubsection{%s}")
)
)
  • The following file, ~/.emacs.d/org-templates/header-foiltex.org contains useful global definitions:
#+AUTHOR: Pierre Hily-Blant
#+email: pierre.hily-blant@univ-grenoble-alpes.fr
#+LaTeX_CLASS: foils
# +LaTeX_CLASS_OPTIONS: [footrule, 17pt, Screen16to9]
#+LaTeX_CLASS_OPTIONS: [footrule, 17pt, Screen4to3]
#+LATEX_HEADER: \MyLogo{Redefine MyLogo in LATEX\_HEADER}
#+LATEX_HEADER: \setlength{\parindent}{0cm}
#+LATEX_HEADER: \usepackage{array}
#+LATEX_HEADER: \usepackage[dvipsnames]{xcolor}
#+LATEX_HEADER: \newcolumntype{C}[1]{>{\centering\arraybackslash}p{#1}}
#+LATEX_HEADER: \renewcommand\labelitemi{$\rhd$}
# +LATEX_HEADER: \renewcommand\labelitemi{$\square$}
#+LATEX_HEADER: \renewcommand\labelitemii{$\bullet$}
# +LATEX_HEADER: \renewcommand\labelitemii{{\normalfont\bfseries --}}
#+LATEX_HEADER: \renewcommand\labelitemiii{{\normalfont\bfseries --}}
#+LATEX_HEADER: \usepackage{natbib}
#+LATEX_HEADER: \usepackage{pxfonts}
#+LATEX_HEADER: \usepackage[version=3]{mhchem}
#+LATEX_HEADER: \setlength{\foilheadskip}{-30pt}
#+LATEX_HEADER: \setlength{\headheight}{-10pt}
#+LATEX_HEADER: \setlength{\headsep}{-10pt}
#+LATEX_HEADER: \setlength{\textheight}{\paperheight}
#+LATEX_HEADER: \addtolength{\textheight}{-2in}
#+LATEX_HEADER: \newcommand\stress[1]{{\color{red}#1}}
#+LaTeX_HEADER: \hypersetup{linktoc = all, colorlinks = true, urlcolor = MidnightBlue, citecolor = MidnightBlue, linkcolor = orange}
# +latex_header: \usepackage[colorlinks=true,citecolor=electricblue,linkcolor=MidnightBlue]{hyperref}
#+OPTIONS: toc:nil
#+BIND: org-latex-images-centered nil
  • In addition to this global definition file, I found it useful to define a local file, which I called header.org and located in the directory containing my org files. This header.org file gathers all options common to my various presentations, to ensure consistency throughout a entire Lecture.
#+setupfile: ~/.emacs.d/org-templates/header-foiltex.org
# +TITLE: Stars: formation, structure, evolution
#+LATEX_HEADER: \MyLogo{Physical Fluid Dynamics}
# +AUTHOR: Pierre Hily-Blant (University Grenoble Alpes/IPAG)
# +email: pierre.hily-blant@univ-grenoble-alpes.fr
#+LATEX_HEADER: \MyLogo{Stars: formation, structure, evolution}
#+LATEX_HEADER: \input{general.tex}
#+LATEX_HEADER: \input{../book/STELLAR_shortcuts.tex}
  • Finally, an example of org-mode file could be
#+SETUPFILE: ./header-stellar.org
#+title: Demo
#+subtitle: From protostars to stars
#+DATE: 17-nov-2021
* Example with a single image.
  #+attr_latex: :center t :height .5\textheight
  [[./figs/TTauri-joy1945.png]]
** Example with two images side by side
   #+attr_latex: :align C{0.4\textwidth}C{0.55\textwidth}
  | [[./figs/ET-1msol-overview.png]] | [[./figs/ET-5Msun-iben1991-annot.png]] |
*** A subsubsection
    - with some items
    - written the usual way


    \newpage
    *Start of a new page*
    - with some items or wathever, such as an equation
      \[
      f(x) = int_0^a h(u) du
      \]

Shell

I don't know much about shell commands. Yet, with very few commands, many highly complex operations can be done very fast on large files. It is usually much more efficient to use shell commands from scripts (particularly python which is so slow) to work on files. In this regard, awk and sed are must-know commands.

Loops

  • Slicing
    • You can slice the input using ${@:3} or ${@:3:8} and then loop over it. For instance, to print arguments starting from 3

      for i in ${@:3} ; do echo $i; done
      

      or to print 8 arguments starting from 3 (so, arguments 3 through 10)

      for i in ${@:3:8} ; do echo $i; done
      
  • While loop

    i=1; while test $i -le 60; do let n=8000+$i ; echo $n ; let i=$i+1; done
    

Managing Strings

  • Change a generic part of filenames
    • e.g. change the 'file1' string into 'newfilename' in all files called file1.*.

      rename file1 newfilename file1.*
      

      is equivalent to

      for f in file1.*; do mv "$f" "${f/file1/newfilename}"; done
      

      You can also use regular expressions:

      rename 's/titi/toto/' tutu*
      rename 'y/a-z/A-Z/' *
      
  • Substitutions

    a=toto.jpeg
    b=jpeg
    c=gif
    echo ${a/$b/$c}
    toto.gif
    $ echo ${a/jpeg/gif}
    toto.gif
    
  • Substitutions using grep
    • Simple substitutions

      grep -e "subroutine\|intent\|\ function\ " tasks/grid_class.f90
      grep -v ^.*\! lib/*f90 | grep [$]
      
    • To exclude lines beginning with 0 or more blanks, followed by "!"

      grep -v '^\s*\!' spinstate.f90 | wc -l
      
    • Same as above, also excluding empty lines:

      grep -v '^$\|^\s*\!' spinstate.f90 | wc -l
      
  • Extract a given line from a file

    sed "NUMq;d" input_file
    

    where NUM is the line number. In a callable routine with variables:

    sed "${NUM}q;d" $file
    
  • Change the field delimiter
    • The following example shows how to reformat a data table into a Latex table.

      awk '$1=$1' FS=" " OFS=" & " analyse.dat| sed 's/$/\\\\/'
      

grep

  • Regular expressions is so vast… So, only few repetitive examples below.
  • Make this test to see what your shell sees; note the differences between single and double quotes:

    printf '<%s>\n' ref "ref" 'ref' \ref "\ref" '\ref' \\ref "\\ref" '\\ref'
    
  • Find an expression like \ref in a tex file:

    # Single quotes
    grep '\\ref' *.tex | grep ch\: # this works
    grep -F '\\ref' *.tex | grep ch\: # this does NOT work
    grep '\ref' *.tex | grep ch\: # this does NOT works
    grep -F '\ref' *.tex | grep ch\: # this does work
    # Double quotes
    grep "\ref" *.tex | grep ch\: # this does NOT work
    grep "\\ref" *.tex | grep ch\: # this does NOT work
    grep "\\\ref" *.tex | grep ch\: # this does work
    grep -F "\ref" *.tex | grep ch\: # this works
    grep -F "\\ref" *.tex | grep ch\: # this works
    grep -F \\ref *.tex | grep ch\: # this works
    
  • Chose the one you like.

Sed

  • Read and print a file until a PATTERN is matched:

    sed '/PATTERN/q' filename
    

    will print up to and including the match

    sed '/PATTERN/Q' filename
    

    print up to BUT NOT including the match. See also with Awk.

  • More about printing a range of a file:

    sed -n '1,/<pattern>/ p' <file> # from the first line until pattern is matched
    sed -n '/<pattern>/,$ p' <file> # from pattern to end of file
    sed -n '/<pattern1>/,/<pattern2>/ p' <file> # from pattern1 to pattern2     
    
  • Typical example for Latex formatting:

    sed 's/$/\\/' toto.sh
    
  • Suppress carriage return

    sed ':a;N;$!ba;s/\n/ /g' < input > output
    

Awk

  • Useful resources here and here
  • If, then, else:

    awk '{if (NF==1) {print $1} else print $0}' < tdv.tex
    
  • Extract part of a file (see also with Sed)

    awk '{print} /pattern/ {exit}' filename #from beginning to line matching 'pattern', included
    awk '/pattern/ {exit} {print}' filename #from beginning to line matching 'pattern', excluded
    
  • For loops

    for ((xx=4;xx<13;xx+=1)) ; do awk -v x=$xx 'BEGIN {printf "%6.1f",x/10.}  ($5 == x/10.) {printf "& %12.2e &%10.2e",$6,$7} END {printf "\\\\"}' < grid_abinit.log ; echo ; done
    awk '($1 !~ /^!/) {print $0}' <terzieva2000.dat
    awk '{ for(i = 1; i <= NF; i++) { print i; } }' Grid_zeta/sstate.header
    awk '{ for(i = 1; i <= NF; i++) { print $i; } }' Grid_zeta/sstate.header
    
  • Print a substring:

    awk '{print $1,"=",substr($0,index($0,$2))}' < ~/.gnuplot.d/global.cfg
    

    will print the first field, followed by =, and the remaining of the input line; for more info on substrings etc, look at this link.

  • Texify a data table:

    awk 'BEGIN {OFS=" & ";ORS="\\\\\n"} {$1=$1} 1' opacity_one.dat
    

Find

  • Find files older than a given file:

    find . ! -cnewer ./yourfile -print
    
  • Find files and tar them:

    find / -type f -mtime -7 | xargs tar -rf weekly_incremental.tar
    

Pipe and more

  • To send, line by line, the output of a command:

    grep "png" stellar-1.org | sed -e 's/\[//g' -e 's/\]//g' -e 's/\.\//\.\.\/figs\//' | xargs -n1 ls
    
  • See here for a detailed explanation of xargs

Managing files

Dealing with large tar files

pigz -dc target.tar.gz | tar tf -
pigz -dc target.tar.gz | tar xf - 'filename'

Change ^M from MAC to \n in UNIX:

  • with tr

    for file in `ls` ; do tr '\r' '\n' < $file > dummy.txt ; mv dummy.txt $file ; done
    
  • This can also be done using sed:

    sed -i 's/^M/\n/g' filename
    

Suppress carriage return

  • with sed

    sed ':a;N;$!ba;s/\n/ /g' < input > output
    
  • with tr

    tr -d '\n' < input
    

Columns

  • Append files by columns

    paste file1.dat file2.dat
    paste file?.dat
    
  • Reformat files with constant column widths

    for file in `ls *txt` ; do column -t $file > dummy.txt ; mv dummy.txt $file ; done
    
  • Extract columns 2-3 from fullgrid_20K.log

    cat fullgrid_20K.log | tr -s ' ' | cut -d' ' -f3-4 > test.dat
    
  • Extract columns 4-5 from fullgrid_20K.log and collate them right to test.dat

    cat fullgrid_20K.log | tr -s ' ' | cut -d' ' -f5-6 | paste test.dat - > test2.dat
    
  • Replace the first 10 columns of fdens_1e6.out by columns 3-END of fullgrid_20K.log

    cat fullgrid_20K.log | tr -s ' ' | cut -d' ' -f4- | paste - <(awk '($1 !~/^\#/) {print $0}' fdens_1e6.out | tr -s ' ' | cut -d ' ' -f 11-) | sed 's/\ /\ \ /g' > test2.dat   
    

Change the field delimiter

  • One usual example is to transform an ASCII table of numbers into a LATEX table.

    awk '$1=$1' FS=" " OFS=" & " < input_file | sed 's/$/\\\\/'
    

    FS and OFS are awk special variables which means Input Field separator and Output field separator respectively. $1=$1 actually does nothing. For awk, to change the delimiter, there should be some change in the data and hence this dummy assignment.

Miscellaneous

tar --atime-preserve -c -v -z -f presentations.tarz Presentations
a2ps -M A4 --column=1 --font-size=10 check.dat -o check.ps

Converting images

  • Use the (fantastic) ImageMagik tools collection

    convert [input-option] input-file [output-option] output-file
    

    for instance:

    convert input.eps -quality 100 output.jpg
    
  • In some cases, some files produced by GhostScript are protected against conversion and one obtains the error message below:

    convert-im6.q16: attempt to perform an operation not allowed by the security policy `PS' @ error/constitute.c/IsCoderAuthorized/421.
    convert-im6.q16: no images defined `05aug07b_mapw.png' @ error/convert.c/ConvertImageCommand/3229.
    
    • The reason is a bug in version earliers (strictly) than 9.24 of gs
    • Solution: with version ≥ 9.24 of gs, use gs to perform the conversion

      gs -r200 -sDEVICE=jpeg -o output.jpg input.eps
      

      where -r<res> pixels/inch resolution and type gs -h to get a list of the accepted output formats.

System commands

  • This command will usually tell you the vendor and model of your card. Open xterminal or shell prompt and type the command:

    lspci
    lspci -v
    lspci -v | less
    
  • To check loaded keys (for eg ssh)

    ssh-add -L
    ssh-rsa AAAAB....
    

Resize an image with convert

$ identify -format "%wx%h\n" test.jpg
1280x623
$ convert test.jpg -resize 640x312\> titi.jpg
$ identify -format "%wx%h\n" titi.jpg
640x312

zsh

  • changing the colours of the ls command output (see here). Put this into the .zshrc (or .bashrc) file:

    LS_COLORS=$LS_COLORS:"ln=01;31;43"
    alias ll='\ls -l --color=auto'
    

Gnuplot

Gnuplot is the greatest plotting I found, especially since v5. Possibilities are huge, scripts are compact, commands are sufficiently simple and consistent to be memorized. In addition, since v5, the export to high-quality plots, which was one of the reasons I left gnuplot for years, is now at a high level, especially through the use of the cairo libraries. A particularly useful resource is the book "Gnuplot in action". Another one, on-line resource is http://www.gnuplotting.org

Plotting

  • Formatting

    set format y "%g"
    set format y "10^{%L}"
    
  • Grid: an example to plot a grid at major and minor x and y tics in a log scale.

    set mxtics 10
    set mytics 10
    set grid mxtics
    set grid mytics
    set grid lt 8 lw 2, lt 8 lw 1
    
  • set offsets: to add space left/right (top/bottom) to avoid having the first and last points on the border; however, However, the margin's size is not the input value in units of the axis; instead the value is rounded to the next auto-generated tic: set autoscale xfix can be used to enforce that inner margins are in unis of the selected axis.
  • x-axis labels; xtic() allows to build the labels of the x-axis directly from the values in given column of the data file; the following command would plot column 11 × 1000 with the row number as x (column 0), and with x-axis labels corresponding to the concatenation of the columns 3 and 4. The labels are rotated by 60 degrees, centered, and shifted downwards by 1 character.

Scripting

  • The load command loads a gnuplot script file; no arguments

    load 'toto.gnu'
    
  • The call command allows up to 9 arguments; the syntax in the script is like this:

    filename = ARG1
    if (ARGC >= 2){tkin=ARG2}
    if (ARGC == 3) {set title ARG3 offset 0,-5 font ",20"}
    

    and the calling sequence is

    call 'toto.gnu' "toto.dat" 10 "TOTO"
    
  • Using MACROS: this is a very useful, although somewhat limited, tool; the following files are located in $HOME/.gnuplot.d/:
    • put the following png.cfg file:

      fileout=ARG2
      if (ARGC == 0) {print "Usage: @PNG <font_size> [<fileout>]" ; exit(0)} else {fileout=ARG2}
      if (ARGC < 2) {fileout="test.png"} else {fileout=ARG2}
      sfont=sprintf("\"Carlito,%s\"",ARG1)
      print sfont
      set term push
      set term pngcairo \
          notransparent \
          enhanced \
          font @sfont \
          fontscale 1 \
          rounded \
          crop \
          size 800,800
      set output fileout
      repl
      set output
      print "Output: ",fileout
      set term pop
      
    • in the general file general.cfg:

      set macro
      PNG = "call 'png.cfg'"
      
    • now, from gnuplot command line, typing the folling command will produce toto.png file using Carlito,18 font type and size:

      @PNG 18 "toto.png"
      

Loops, iterations, etc

  • In the following example, we iteratively plot the weight of each species, between column 74 to 83, relative to their sum, and label each curve by the name of the species with the column number concatenated to it

    plot [1e3:*] [1e-10:*] for [j=74:83] 'output/fdens_speci.out' u 2:(column(j)/(sum[k=74:83] column(k))) t columnhead(j).j
    
  • Here, we iteratively plot the weight of species taken from a list relative to the sum of all species corresponding to columns 74 to 83, and put a legend accordingly.

    plot [1e3:*] [1e-10:*] for [s in "CO NH3 H2O"] 'output/fdens_speci.out' u 2:(column(s)/(sum[k=74:83] column(k))) t s
    
  • Just checking this works really:

    plot [1e3:*] [1e-10:*] for [s in "CO NH3 H2O"] 'output/fdens_speci.out' u 2:(column(s)/(sum[k=74:83] column(k))) t s, \
    '' u 2:($18/($75+$83+$74+$77+$96+$78)) every 10 w p t "Checking"
    

    Here, we have overplotted the H2O species, with points, taken every 10 samples to make it easy to read. See the result below:

    gnuplot_demo_1.png

  • A loop over three files:
  • A script taken from the "Gnuplot in Action" book:
  • Operation on columns within a loop

    s14 = "N N2 CN NO NH3 HCN"
    s15 = "N. N.N CN. N.O N.H3 HCN."
    set border 3
    ratio(k)=column(word(s14,k))/column(word(s15,k))
    fac(x) = x==2?2:1 # used to multiply by 2 if N2
    plot for [k=1:words(s14)] 'grid_tkin_SS.out' u 3:(ratio(k)*fac(k))
    

Labelling

  • This example would put an opaque box underneath the legend, shifting leftwards the symbols towards the text by 3 character units.

    set key at graph 0.1,0.95,0 left Left width -3 height 1 box opaque
    
  • The following puts thick dots at given points and label them A to C (here, orl(x,y) is a user-defined function):

    roots = "267.9 286.0 306.8" # for S = 1365 W m-2
    names = "A B C"
    r2=252.27
    r3=317.95
    unset label
    do for [i=1:3] {
    	   x = word(roots,i)
    	   ts = 1.*x
    	   set label word(names,i) at x,orl(x,0.8) point ps 2.5 pt 7 right offset -1,1
    	   cc=sprintf("%4s%12.1f%8.1f%12.1f", word(names,i),ts,ts-273.15,orl(x,0.8))
    	   print cc
           }
    

Exporting plots

  • pdfcairo: produces a PDF file; but not cropped… And issues with the size of the symbols: the size in the PDF does not reflect that in the window.
  • export to PNG:
  • An alternative, which allows to control directly the DPI: export as SVG and use in-line command inkscape to export to lossless PNG:

    set term svg enhanced size 500,500
    set output 'test.svg' ; repl ; set output
    inkscape --export-area-drawing --export-png=test.png --export-dpi=300 --export-background-opacity=1 test.svg
    
  • See alse here for some useful details.

Exporting outputs

  • To write the values in ASCII format:

    set table "tsurf.tex" separator "&"
    plot [0.1:50] [10:1000] 'tsurf.dat' \
         u (sprintf("%4.2f",$1)) : (sprintf("%4.2f",$2)) : (sprintf("%6.2f",ts($1,0))) with table
    unset table
    

Installation

  • By default, Debian install does not include the +LIBREADLINE; use show version long in gnuplot to see the compilation options;
  • You have to install it manually…
  • The following is taken from here.
  • Download the source:
    • You need to have the deb-src in the /etc/apt/sources.list file;
    • then

      cd /usr/local/src/
      sudo apt-get source gnuplot
      sudo apt build-dep gnuplot
      cd /usr/local/src/gnuplot-*.*.*/debian/
      
    • Edit the file rules by adding:

      --with-readline=gnu --enable-history-file
      
    • Then go up and compile:

      cd ..
      sudo dpkg-buildpackage
      
    • I noticed some loops when compiling the doc (as it seems to be) with images which are missing. Just wait…
    • Then, install the package:

      cd ..
      sudo dpkg -i gnuplot-*.deb
      
    • Exclude gnuplot from the update:

      ➜  src sudo apt-mark hold gnuplot
      gnuplot set on hold.
      

Image/PDF files

  • The basic tools are from pdftk and imagemagick
  • Concatenate PDF files

    man pdftk
    pdftk A=in1.pdf B=in2.pdf cat A1-end B3-end output test.pdf
    
  • Clip an image:

    convert -trim input.png output.png
    
  • Crop and clip an image: the following would split an image with four panels.

    fig="input.png"
    ww=`identify -format "%[fx:w] %[fx:h]\n" $fig | awk '{print $1}'`
    hh=`identify -format "%[fx:w] %[fx:h]\n" $fig | awk '{print $2}'`
    mx=`expr $ww / 2`
    my=`expr $hh / 2`
    #echo $ww $mx $hh $my
    convert $fig -crop -$mx-$my -trim topleft.png
    convert $fig -crop +$mx-$my -trim topright.png
    convert $fig -crop -$mx+$my -trim bottomleft.png
    convert $fig -crop +$mx+$my -trim bottomright.png
    exit 0
    
  • Crop a PDF file:

    pdfcrop -clip input.pdf output.pdf
    

Git

References

Basic commands

  • Advice: to be exectued every time you're leaving a session, no matter whether this is finished or not. The repository will thus behave as a backup. Obviously, do this in the dev branch, or in the master branch if this is version 0.x.

    git br # shows in which branch you are currently
    
  • To see what has changed; you have usually two lists:
    • The first lists the files registered in the repository that have been modified since the last commit.
    • The second lists unregistered files
  • To see the changes

    git diff <filename>
    
  • To make the commit:

    git status
    git add -u <files> # list of files you will commit
    # git add *.f90 # would do what you think
    git commit -m "a message" # to commit the changes
    git push
    
  • Note that, if this is the first commit, then you have to do:

    git push origin <new-branch-name> # to initialise the new branch with your version of the code
    
  • To revert a local file to the version in the repository:

    git checkout HEAD <filename>
    
  • Create aliases

    $ git config --global alias.co checkout
    $ git config --global alias.br branch
    $ git config --global alias.st status     
    

Working with branches

  • This is essential. And this is, for me, the incredible plus of git to make such complex things so easy.
  • Create a branch

    git co -b <new_branch>		# to create a new branch
    
  • To create the branch on the remote server:

    $ git push origin <new_branch>
    Total 0 (delta 0), reused 0 (delta 0)
    To git+ssh://git.renater.fr:2222/scmrepos/git/chemcollapse/chemcollapse.git
     * [new branch]      test -> test
    
  • Pushing files from a branch for the first time:

    ➜  chemcollapse git:(dev) ✗ git push
    fatal: The current branch dev has no upstream branch.
    To push the current branch and set the remote as upstream, use
    
    git push --set-upstream origin dev
    
  • Moving from one branch to another

    git br		# to see where you are
    git co branch_name # to move to another branch
    
  • Initiate a branch: the point to be understood is that you will create a branch starting from an existing ones. For example, creating a branch dev should be done like this:

    $ git br		# check that you are in master
    $ git co -b dev	# create the branch dev
    $ git br		# chech that you are in dev
    

    From this point, you are starting a dev branch from the current version of the master branch.

  • Example, with a branch dedicated to improve the reading of the input parameters of ALICO:

    $ git push --set-upstream origin read_input_parameters
    Counting objects: 22, done.
    Delta compression using up to 4 threads.
    Compressing objects: 100% (22/22), done.
    Writing objects: 100% (22/22), 22.47 KiB | 0 bytes/s, done.
    Total 22 (delta 12), reused 0 (delta 0)
    To ssh://git.renater.fr:2222/alico.git
    * [new branch]      read_input_parameters -> read_input_parameters
    Branch read_input_parameters set up to track remote branch read_input_parameters from origin.
    

    The option set-upstream tells git to push from our local read_input_parameters branch the one with the same name in the repository. This needs to be done only for the first push. After this, a simple git push is sufficient.

  • To delete a local or remote branch

    git branch -d the_local_branch		# To delete a local branch:
    git push origin --delete the_remote_branch # To delete a remote branch:
    

Conflicts

  • Resolving conflicts: see here

Problem with branches

  • Your branch and 'origin/master' have diverged, # and have 1 and 1 different commit(s) each, respectively.
    • Taken from this thread
    • check if you need to update origin.
      • If origin is up-to-date, then some commits have been pushed to origin from another repo while you made your own commits locally:

        ... o ---- o ---- A ---- B  origin/master (upstream work)
        		   \
        		    C  master (your work)
        
      • As can be seen: commit C is based on commit A because that was the latest work you had fetched from upstream at the time.
      • However, before trying to push back to origin, someone else pushed commit B. Development history has diverged into separate paths.
      • You can then merge or rebase. See Pro Git: Git Branching - Rebasing for details.
    • Merging (not my prefered solution)
      • Use the git merge command:

        $ git merge origin/master         
        
      • This tells Git to integrate the changes from origin/master into your work and create a merge commit. The graph of history now looks like this:

        
        ... o ---- o ---- A ---- B  origin/master (upstream work)
        		   \      \
        		    C ---- M  master (your work)
        
      • The new merge, commit M, has two parents, each representing one path of development that led to the content stored in that commit. Note that the history behind M is now non-linear.
    • Rebase (my prefered solution)
      • Use the git rebase command:

        $ git rebase origin/master
        
      • This tells Git to replay commit C (your work) as if you had based it on commit B instead of A. CVS and Subversion users routinely rebase their local changes on top of upstream work when they update before commit. Git just adds explicit separation between the commit and rebase steps.
      • The graph of history now looks like this: You can review the differences with a:

        ... o ---- o ---- A ---- B  origin/master (upstream work)
        			  \
        			   C'  master (your work)
        
      • Commit C' is a new commit created by the git rebase command. It is different from C in two ways:
        1. It has a different history: B instead of A;
        2. Its content accounts for changes in both B and C; it is the same as M from the merge example.
      • Note that the history behind C' is still linear which makes it my prefered choice. This approach preserves the CVS-based workflow used previously and may ease the transition. An attempt to push C' into our repository will work (assuming you have permissions and no one has pushed while you were rebasing).
      • The git pull command provides a shorthand way to fetch from origin and rebase local work on it:

        $ git pull --rebase
        

        This combines the above fetch and rebase steps into one command.

Filesystem

Mounting

  • basics

    cd /media/username/
    sudo mkdir DESTINATION
    sudo mount /dev/XXX DESTINATION
    

    where the partition id /dev/XXX may be obtained using the lsblk command (see here) or fdisk -l (see also below)

  • General reference here
  • Another reference here

    The correct syntax is:
    
    sudo mount -o remount,rw /partition/identifier /mount/point
    
    Where mount/point is /partition/identifier's corresponding mountpoint,
    as listed by the following command:
    mount -v | grep "^/" | awk '{print "\nPartition identifier: " $1  "\n Mountpoint: "  $3}'
    
  • To identify the UUID (a unique partition identifier):

    ➜  ~ sudo blkid
    /dev/mapper/ipag--9006--vg-root: UUID="e963b1f8-17a4-44cf-8a7b-23f5eed42c67" BLOCK_SIZE="4096" TYPE="ext4"
    /dev/nvme0n1p3: UUID="cf67d2b6-7ea6-456a-a8df-d9bcdf504f81" TYPE="crypto_LUKS" PARTUUID="ab456709-6596-4762-a607-06641e8eea4f"
    /dev/nvme0n1p1: UUID="63B5-6AAC" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="d9a32a12-fea5-4d58-93a0-29b967961709"
    /dev/nvme0n1p2: UUID="ba818f17-62b6-4d3e-83d3-5706ea6e07c3" BLOCK_SIZE="1024" TYPE="ext2" PARTUUID="370265e2-70d1-4d37-9b20-ea9f11516156"
    /dev/mapper/ipag--9006--vg-swap_1: UUID="7449b68c-6993-4bf6-a687-6b5157881a74" TYPE="swap"
    /dev/mapper/nvme0n1p3_crypt: UUID="1oKKbs-agJx-aful-nSnK-0HF5-wzqf-UKBYRd" TYPE="LVM2_member"
    /dev/sda1: LABEL="phb2000" UUID="d945144b-47d2-4f42-b996-9b20b5772115" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="36a79383-28e1-446d-9fa5-a68c1c716ddf"
    
  • Set permissions of a USB drive upon mounting

    ➜  ~ mount         
    /dev/sda1 on /media/hilyblap/phb2000 type ext4 (rw,nosuid,nodev,relatime,errors=remount-ro,uhelper=udisks2)
    

Repairing a filesystem

  • Basic tool is fsck; see here
  • How to deal with Can't read superblock: see here

RAID system

  • See here
  • How to deal with /dev/sdb is apparently in use by the system

    umount /dev/md0
    mdadm --stop /dev/md0
    mdadm --zero-superblock /dev/sdb # clear superblock
    mdadm --remove /dev/md0 # remove array
    

Global installation

Debian

  • apt
    • Useful links
    • update and upgrade
      • The following packages have been kept back
        • Means that some package could not be upgraded; this could be due to several reasons; usually, this is because of dependency issues; most common issues are
          • the new package relies on a package for which there is no update in the current distro
          • the new package rests upon new package not installed in the current distro
        • To fix the issue, try:

          sudo apt-get install the_problematic_package
          
  • How to know which distribution is installed?

    cat /etc/os-release
    

    produces the following output:

    PRETTY_NAME="Debian GNU/Linux 10 (buster)" NAME="Debian GNU/Linux" VERSION_ID="10" VERSION="10 (buster)" VERSION_CODENAME=buster ID=debian HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support" BUG_REPORT_URL="https://bugs.debian.org/"

Gnome

  • DVD ripper: Handbrake
  • Calendar To set Monday as the first weekday ?

    sudo cp /usr/share/i18n/locales/en_US /usr/share/i18n/locales/en_US.bak
    sudo emacs -nw /usr/share/i18n/locales/en_US
    

    To this file, add the following line:

    first_weekday 2
    

    The logout and login.

Sound-Recorder:

  • Comment definir l'entree pour enregistrement
    • open sound-recorder
    • open PulseAudio Volume Control (PAVC)
    • start recording in sound-recorder
    • set the input from the menu un the PAVC

internal –> internal microphone monitor of internal –> carte son

Bluetooth

ls /run/user/1000/gvfs/obex…

PDF viewer

  • qpdfview
    • View toolbar: zoomIn,scaleFactor,zoomOut,separator,continuousMode,twoPagesMode,separator,rotateLeft,rotateRight,separator,fullscreen,presentation

Softwares, other

Python

  • Use python3 instead of python

ADS

  • Positional arguments: see this link
  • Example
    • pos(author:”Oort, J”,2)papers which have “J. Oort” as the second author
    • pos(author:”Oort, J”,1,3)papers which have “J. Oort” as first, second, or third author

WPS

  • Add a dictionnary

    sudo apt-get install hunspell-fr-comprehensive
    cd /opt/kingsoft/wps-office/office6/dicts/spellcheck
    sudo ln -s /usr/share/hunspell/fr_FR.aff main.aff
    sudo ln -s /usr/share/hunspell/fr_FR.dic main.dic
    cat /opt/kingsoft/wps-office/office6/dicts/spellcheck/fr_FR/dict.conf
    [Dictionary]
    DisplayName=French (Comprehensive)
    DisplayName[de_DE]=French (Comprehensive)
    
  • R1C1 reference style: Menu -> Options -> General and save

Libreoffice

  • To Start With a Defined Page Number:
    1. Click into the 1st paragraph of the page you want to set the number
    2. Format -> Paragraph -> Text flow.
    3. In the Breaks area, enable Insert and With Page Style. Set the page number.

Gimp

  • Put a white background in a transparent png
    • open png
    • Layer -> New layer: chose backgroud layer
    • The white layer is above the png file
    • Layer > Stack -> Reverse layer order
    • File -> Export…

Conky

This is system monitor for X. See here.

Last Updated .