Tips and tricks: the non-geek scientist
Table of Contents
Emacs
Needless to say how powerful emacs
is. However, it can quickly
become criptic. The Practical Emacs Tutorial by Xah Lee is an
excellent reference for those like me who already know something,
but very little.
Miscellaneous
- hidden files:
.class
are not showing with TAB-completion; customize variablecompletion-ignored-extensions
Make programming easy for a non-geek scientist
There are many tools to develop complex codes, so-called IDEs (Integrated Development Environment). However, I tend to try and use as little CPU and power resources as possible, and therefore, try to minimize the number of softwares. Actually, developping with emacs is more than sufficient: it is extremely efficient. I have been developping codes with more than 1e5 lines with emacs. However, I am not a professional code developper and therefore, the following may only be of interest for scientists.
Latex code folding via outline
minor mode
Adding the following lines to my .emacs.d/init.el
allows to
circulate through the Latex file. This is based on this reference.
(add-hook 'prog-mode-hook 'outline-minor-mode) (global-set-key [M-left] 'outline-hide-body) (global-set-key [M-right] 'outline-show-all) (global-set-key [M-up] 'outline-previous-heading) (global-set-key [M-down] 'outline-next-heading) (global-set-key [M-S-left] 'outline-hide-subtree) (global-set-key [M-S-right] 'outline-show-subtree)
Fortran code
Aim: have something as simple as org-mode
to circulate within a
program (e.g. Fortran). After spending some time, and thanks to
brilliant emacs
users, I found a minimal, though, sufficient,
setting that does not need any additional package. This applies to
Fortran
, Latex
codes, etc.
In my .emacs.d/init.el
, I added the following:
;; this is from: https://www.reddit.com/r/emacs/comments/e2u5n9/code_folding_with_outlineminormode/ (add-hook 'prog-mode-hook 'outline-minor-mode) (defun my-outline-fortran () "Fold definitions in Fortran." (setq outline-regexp (rx (or ;; Module and interface blocks. (group (group (* space)) (or "module" "interface")) ;; Procedures and type definitions. (group (group (* space)) (*? (or "pure " "impure " "elemental ")) (or "function" "subroutine" "interface" "type" "type,") (group (+ space))))))) (add-hook 'f90-mode-hook 'my-outline-fortran) ;; The following is needed if you did not define these keys somewhere else. ;; (global-set-key [M-left] outline-hide-body) ;; (global-set-key [M-right] 'outline-show-all) ;; (global-set-key [M-up] 'outline-previous-heading) ;; (global-set-key [M-down] 'outline-next-heading) ;; (global-set-key [M-S-left] 'outline-hide-subtree) ;; (global-set-key [M-S-right] 'outline-show-subtree)
Editing
- show line number on the left in the window:
M-x linum-mode
- Disable fontification in math mode
M-x customize-variable font-latex-fontify-script <RET>
- Scrolling "a la EXCEL" with a top row that is always visible: the
trick is to split into two buffers, with the upper one containing
only one row:
C-u 1 C-x 2
Replace carriage return with a blank
M-x replace-string C-q C-j ENTER space # C-q is the escape character and C-j is for the newline
Faces
- Check the face of the character below the cursor:
M-x describe-face
: the easiestC-u C-x =
will give many information and a link towards the specific face. If you are in a console or are doing emacs -nw in a terminal, instead of clicking, put the cursor on the appropriate element and press RET.
- Inspect the value of
frame-background-mode
and customize it if needed
Emacs/org-mode
Using the truely fantastic org-mode
tool makes emacs
the most
powerful tool for my everyday life as a scientist and teacher. The
org-mode
saves you a huge amount of time when producing rich
READMEs, web pages
Editing
- Commenting:
C-c ;
to comment an entire entry at once - Narrowing: to isolate an entry, use
C-x n s
andC-x n w
to get back to the full TREE. - Navigation
C-c n/p
: Next/Previous heading, regardless of levelC-c f/b
: Next/Previous heading, same levelC-c u
: to go backward to upper level
- To display all links in plain text:
M-x org-toggle-link-display
- if you want to keep this behavior permanently, simply add
(setq org-descriptive-links nil)
to your init file
- if you want to keep this behavior permanently, simply add
Tables
- Build a table from a tabular text: copy/paste the table, then
select it with
C-x C-x
and thenC-c |
- Operating on columns
- swap two columns:
ALT + left/right
- remove current column:
ALT + SHIFT + left/right
- insert column to the right:
ALT + SHIFT + right
- swap two columns:
- Duplication
SHIFT + ENTER
duplicates the current field in the row below; overwrites the content of the field;- Similarly for rows, with
up
anddown
- Clear cell content:
C-c SPC
C-c -
insert a-----
line below the cursor#+TBLFM
to perform operations on columns of a table; several ones can be used, one per line, to be placed right below the table;C-c C-c
applies the formula; output format controlled as follows:#+TBLFM: $2=9*$1**-2.3;%4.3g
.- To add a row counter:
#+TBLFM: $1=@#
. This would start from row 1. Can be adjusted. - Sorting: put the cursor in the column to be sorted, and type
C-c ^
and chose a value from the menu in the bottom menu bar. C-c }
Toggle the display of row and column numbers for a table, using overlays.C-c +
will compute the sum of the column above.@>$3=vsum(@2..@-1)
@>
means 'last row'$3
is the 3rd columnvsum
is a vector sum; may be used either w/ columns or rows@-1
means last row above the current location, hence penultimate;
- See here or here (orgmode.org) for more, and even more.
Appearance
org-indent-mode
: see here
Links
C-c C-l
to edit the linkM-x org-toggle-link-display
to stwitch between collapsed/extended view of links
Using org-agenda
- General link here
C-c a
to open org-agenda dialog box
Using code blocks
To prevent
org-mode
andbabel
to evaluate code blocks during export, put this at the top of the org document:#+PROPERTY: header-args :eval never-export
Exporting
- To export:
C-c C-e
and then follow the menu - HTML export
C-c C-e P f
will export the file within a projectINFOJS_OPT
must be given once and for all and on a single line#+OPTIONS: num:t
will disable the folding property of infojs
- LATEX export: typical sequence to produce a PDF is
C-c C-e l p
Oral presentations
org-mode
can make it easy to produce oral presentation documents. I
suggest to explore two ways: one is the reveal.js
export which
produces HTML presentations; nice, but quickly criptic. I personally
decided not to go ahead with reveal.js
essentially because it was
quickly becoming lengthy and resembling latex files with a significant
amount of useless characters.
So I moved to using the foiltex
package which will sound completely
stone-age to those who knew it from the 2000s, and probably to anyone
indeed; yet, this produces simple and focused, high quality, PDF
documents. And, in combination with org-mode
, I finally have a way
to produce latex-compatible slides with highly efficient files.
There must be some way to improve on what I did which started from this reference.
- An example of a PDF file produced with this method may be seen here.
- The following lines are adapted from this reference. I added
them to my
~/.emacs.d/myinit.org
.
;; Foiltex with emacs org-mode ;; from https://github.com/tomfaulkenberry/orgFoils (require 'ox-latex) (add-to-list 'org-latex-classes '("foils" "\\documentclass{foils}" ("\\foilhead{\\color{MidnightBlue}\\bf\\huge\\raggedright\\rule{10pt}{3ex}\\hspace{1em} %s}" . "\\section{%s}") ("\\foilhead{\\color{RedOrange}\\Large\\raggedright\\rule{10pt}{2ex}\\hspace{1em} %s}" . "\\subsection{%s}") ("\\foilhead{\\color{Blue}\\large\\raggedright\\rule{1ex}{1ex}\\hspace{1em} %s}" . "\\subsubsection{%s}") ) )
- The following file,
~/.emacs.d/org-templates/header-foiltex.org
contains useful global definitions:
#+AUTHOR: Pierre Hily-Blant #+email: pierre.hily-blant@univ-grenoble-alpes.fr #+LaTeX_CLASS: foils # +LaTeX_CLASS_OPTIONS: [footrule, 17pt, Screen16to9] #+LaTeX_CLASS_OPTIONS: [footrule, 17pt, Screen4to3] #+LATEX_HEADER: \MyLogo{Redefine MyLogo in LATEX\_HEADER} #+LATEX_HEADER: \setlength{\parindent}{0cm} #+LATEX_HEADER: \usepackage{array} #+LATEX_HEADER: \usepackage[dvipsnames]{xcolor} #+LATEX_HEADER: \newcolumntype{C}[1]{>{\centering\arraybackslash}p{#1}} #+LATEX_HEADER: \renewcommand\labelitemi{$\rhd$} # +LATEX_HEADER: \renewcommand\labelitemi{$\square$} #+LATEX_HEADER: \renewcommand\labelitemii{$\bullet$} # +LATEX_HEADER: \renewcommand\labelitemii{{\normalfont\bfseries --}} #+LATEX_HEADER: \renewcommand\labelitemiii{{\normalfont\bfseries --}} #+LATEX_HEADER: \usepackage{natbib} #+LATEX_HEADER: \usepackage{pxfonts} #+LATEX_HEADER: \usepackage[version=3]{mhchem} #+LATEX_HEADER: \setlength{\foilheadskip}{-30pt} #+LATEX_HEADER: \setlength{\headheight}{-10pt} #+LATEX_HEADER: \setlength{\headsep}{-10pt} #+LATEX_HEADER: \setlength{\textheight}{\paperheight} #+LATEX_HEADER: \addtolength{\textheight}{-2in} #+LATEX_HEADER: \newcommand\stress[1]{{\color{red}#1}} #+LaTeX_HEADER: \hypersetup{linktoc = all, colorlinks = true, urlcolor = MidnightBlue, citecolor = MidnightBlue, linkcolor = orange} # +latex_header: \usepackage[colorlinks=true,citecolor=electricblue,linkcolor=MidnightBlue]{hyperref} #+OPTIONS: toc:nil #+BIND: org-latex-images-centered nil
- In addition to this global definition file, I found it useful to
define a local file, which I called
header.org
and located in the directory containing my org files. Thisheader.org
file gathers all options common to my various presentations, to ensure consistency throughout a entire Lecture.
#+setupfile: ~/.emacs.d/org-templates/header-foiltex.org # +TITLE: Stars: formation, structure, evolution #+LATEX_HEADER: \MyLogo{Physical Fluid Dynamics} # +AUTHOR: Pierre Hily-Blant (University Grenoble Alpes/IPAG) # +email: pierre.hily-blant@univ-grenoble-alpes.fr #+LATEX_HEADER: \MyLogo{Stars: formation, structure, evolution} #+LATEX_HEADER: \input{general.tex} #+LATEX_HEADER: \input{../book/STELLAR_shortcuts.tex}
- Finally, an example of
org-mode
file could be
#+SETUPFILE: ./header-stellar.org #+title: Demo #+subtitle: From protostars to stars #+DATE: 17-nov-2021 * Example with a single image. #+attr_latex: :center t :height .5\textheight [[./figs/TTauri-joy1945.png]] ** Example with two images side by side #+attr_latex: :align C{0.4\textwidth}C{0.55\textwidth} | [[./figs/ET-1msol-overview.png]] | [[./figs/ET-5Msun-iben1991-annot.png]] | *** A subsubsection - with some items - written the usual way \newpage *Start of a new page* - with some items or wathever, such as an equation \[ f(x) = int_0^a h(u) du \]
Shell
I don't know much about shell commands. Yet, with very few commands,
many highly complex operations can be done very fast on large
files. It is usually much more efficient to use shell commands from
scripts (particularly python
which is so slow) to work on
files. In this regard, awk
and sed
are must-know commands.
Loops
- Slicing
You can slice the input using ${@:3} or ${@:3:8} and then loop over it. For instance, to print arguments starting from 3
for i in ${@:3} ; do echo $i; done
or to print 8 arguments starting from 3 (so, arguments 3 through 10)
for i in ${@:3:8} ; do echo $i; done
While loop
i=1; while test $i -le 60; do let n=8000+$i ; echo $n ; let i=$i+1; done
Managing Strings
- Change a generic part of filenames
e.g. change the 'file1' string into 'newfilename' in all files called file1.*.
rename file1 newfilename file1.*
is equivalent to
for f in file1.*; do mv "$f" "${f/file1/newfilename}"; done
You can also use regular expressions:
rename 's/titi/toto/' tutu* rename 'y/a-z/A-Z/' *
Substitutions
a=toto.jpeg b=jpeg c=gif echo ${a/$b/$c} toto.gif $ echo ${a/jpeg/gif} toto.gif
- Substitutions using
grep
Simple substitutions
grep -e "subroutine\|intent\|\ function\ " tasks/grid_class.f90 grep -v ^.*\! lib/*f90 | grep [$]
To exclude lines beginning with 0 or more blanks, followed by "!"
grep -v '^\s*\!' spinstate.f90 | wc -l
Same as above, also excluding empty lines:
grep -v '^$\|^\s*\!' spinstate.f90 | wc -l
Extract a given line from a file
sed "NUMq;d" input_file
where NUM is the line number. In a callable routine with variables:
sed "${NUM}q;d" $file
- Change the field delimiter
The following example shows how to reformat a data table into a Latex table.
awk '$1=$1' FS=" " OFS=" & " analyse.dat| sed 's/$/\\\\/'
grep
- Regular expressions is so vast… So, only few repetitive examples below.
Make this test to see what your
shell
sees; note the differences between single and double quotes:printf '<%s>\n' ref "ref" 'ref' \ref "\ref" '\ref' \\ref "\\ref" '\\ref'
Find an expression like
\ref
in a tex file:# Single quotes grep '\\ref' *.tex | grep ch\: # this works grep -F '\\ref' *.tex | grep ch\: # this does NOT work grep '\ref' *.tex | grep ch\: # this does NOT works grep -F '\ref' *.tex | grep ch\: # this does work # Double quotes grep "\ref" *.tex | grep ch\: # this does NOT work grep "\\ref" *.tex | grep ch\: # this does NOT work grep "\\\ref" *.tex | grep ch\: # this does work grep -F "\ref" *.tex | grep ch\: # this works grep -F "\\ref" *.tex | grep ch\: # this works grep -F \\ref *.tex | grep ch\: # this works
- Chose the one you like.
Sed
Read and print a file until a PATTERN is matched:
sed '/PATTERN/q' filename
will print up to and including the match
sed '/PATTERN/Q' filename
print up to BUT NOT including the match. See also with Awk.
More about printing a range of a file:
sed -n '1,/<pattern>/ p' <file> # from the first line until pattern is matched sed -n '/<pattern>/,$ p' <file> # from pattern to end of file sed -n '/<pattern1>/,/<pattern2>/ p' <file> # from pattern1 to pattern2
Typical example for Latex formatting:
sed 's/$/\\/' toto.sh
Suppress carriage return
sed ':a;N;$!ba;s/\n/ /g' < input > output
Awk
- Useful resources here and here
If, then, else:
awk '{if (NF==1) {print $1} else print $0}' < tdv.tex
Extract part of a file (see also with Sed)
awk '{print} /pattern/ {exit}' filename #from beginning to line matching 'pattern', included awk '/pattern/ {exit} {print}' filename #from beginning to line matching 'pattern', excluded
For loops
for ((xx=4;xx<13;xx+=1)) ; do awk -v x=$xx 'BEGIN {printf "%6.1f",x/10.} ($5 == x/10.) {printf "& %12.2e &%10.2e",$6,$7} END {printf "\\\\"}' < grid_abinit.log ; echo ; done awk '($1 !~ /^!/) {print $0}' <terzieva2000.dat awk '{ for(i = 1; i <= NF; i++) { print i; } }' Grid_zeta/sstate.header awk '{ for(i = 1; i <= NF; i++) { print $i; } }' Grid_zeta/sstate.header
Print a substring:
awk '{print $1,"=",substr($0,index($0,$2))}' < ~/.gnuplot.d/global.cfg
will print the first field, followed by
=
, and the remaining of the input line; for more info on substrings etc, look at this link.Texify a data table:
awk 'BEGIN {OFS=" & ";ORS="\\\\\n"} {$1=$1} 1' opacity_one.dat
Find
Find files older than a given file:
find . ! -cnewer ./yourfile -print
Find files and tar them:
find / -type f -mtime -7 | xargs tar -rf weekly_incremental.tar
Pipe and more
To send, line by line, the output of a command:
grep "png" stellar-1.org | sed -e 's/\[//g' -e 's/\]//g' -e 's/\.\//\.\.\/figs\//' | xargs -n1 ls
- See here for a detailed explanation of
xargs
Managing files
Dealing with large tar files
pigz -dc target.tar.gz | tar tf - pigz -dc target.tar.gz | tar xf - 'filename'
Change ^M from MAC to \n in UNIX:
with
tr
for file in `ls` ; do tr '\r' '\n' < $file > dummy.txt ; mv dummy.txt $file ; done
This can also be done using
sed
:sed -i 's/^M/\n/g' filename
Suppress carriage return
with
sed
sed ':a;N;$!ba;s/\n/ /g' < input > output
with
tr
tr -d '\n' < input
Columns
Append files by columns
paste file1.dat file2.dat paste file?.dat
Reformat files with constant column widths
for file in `ls *txt` ; do column -t $file > dummy.txt ; mv dummy.txt $file ; done
Extract columns 2-3 from fullgrid_20K.log
cat fullgrid_20K.log | tr -s ' ' | cut -d' ' -f3-4 > test.dat
Extract columns 4-5 from fullgrid_20K.log and collate them right to test.dat
cat fullgrid_20K.log | tr -s ' ' | cut -d' ' -f5-6 | paste test.dat - > test2.dat
Replace the first 10 columns of fdens_1e6.out by columns 3-END of fullgrid_20K.log
cat fullgrid_20K.log | tr -s ' ' | cut -d' ' -f4- | paste - <(awk '($1 !~/^\#/) {print $0}' fdens_1e6.out | tr -s ' ' | cut -d ' ' -f 11-) | sed 's/\ /\ \ /g' > test2.dat
Change the field delimiter
One usual example is to transform an ASCII table of numbers into a LATEX table.
awk '$1=$1' FS=" " OFS=" & " < input_file | sed 's/$/\\\\/'
FS and OFS are awk special variables which means Input Field separator and Output field separator respectively. $1=$1 actually does nothing. For awk, to change the delimiter, there should be some change in the data and hence this dummy assignment.
Miscellaneous
tar --atime-preserve -c -v -z -f presentations.tarz Presentations a2ps -M A4 --column=1 --font-size=10 check.dat -o check.ps
Converting images
Use the (fantastic) ImageMagik tools collection
convert [input-option] input-file [output-option] output-file
for instance:
convert input.eps -quality 100 output.jpg
In some cases, some files produced by
GhostScript
are protected against conversion and one obtains the error message below:convert-im6.q16: attempt to perform an operation not allowed by the security policy `PS' @ error/constitute.c/IsCoderAuthorized/421. convert-im6.q16: no images defined `05aug07b_mapw.png' @ error/convert.c/ConvertImageCommand/3229.
- The reason is a bug in version earliers (strictly) than 9.24 of
gs
Solution: with version ≥ 9.24 of
gs
, use gs to perform the conversiongs -r200 -sDEVICE=jpeg -o output.jpg input.eps
where
-r<res> pixels/inch resolution
and typegs -h
to get a list of the accepted output formats.
- The reason is a bug in version earliers (strictly) than 9.24 of
System commands
This command will usually tell you the vendor and model of your card. Open xterminal or shell prompt and type the command:
lspci lspci -v lspci -v | less
To check loaded keys (for eg ssh)
ssh-add -L ssh-rsa AAAAB....
Resize an image with convert
$ identify -format "%wx%h\n" test.jpg 1280x623 $ convert test.jpg -resize 640x312\> titi.jpg $ identify -format "%wx%h\n" titi.jpg 640x312
zsh
changing the colours of the
ls
command output (see here). Put this into the.zshrc
(or.bashrc
) file:LS_COLORS=$LS_COLORS:"ln=01;31;43" alias ll='\ls -l --color=auto'
Gnuplot
Gnuplot
is the greatest plotting I found, especially since
v5. Possibilities are huge, scripts are compact, commands are
sufficiently simple and consistent to be memorized. In addition,
since v5, the export to high-quality plots, which was one of the
reasons I left gnuplot
for years, is now at a high level,
especially through the use of the cairo
libraries. A particularly
useful resource is the book "Gnuplot in action". Another one,
on-line resource is http://www.gnuplotting.org
Plotting
Formatting
set format y "%g" set format y "10^{%L}"
Grid: an example to plot a grid at major and minor x and y tics in a log scale.
set mxtics 10 set mytics 10 set grid mxtics set grid mytics set grid lt 8 lw 2, lt 8 lw 1
set offsets
: to add space left/right (top/bottom) to avoid having the first and last points on the border; however, However, the margin's size is not the input value in units of the axis; instead the value is rounded to the next auto-generated tic:set autoscale xfix
can be used to enforce that inner margins are in unis of the selected axis.- x-axis labels;
xtic()
allows to build the labels of the x-axis directly from the values in given column of the data file; the following command would plot column 11 × 1000 with the row number as x (column 0), and with x-axis labels corresponding to the concatenation of the columns 3 and 4. The labels are rotated by 60 degrees, centered, and shifted downwards by 1 character.
Scripting
The
load
command loads agnuplot
script file; no argumentsload 'toto.gnu'
The
call
command allows up to 9 arguments; the syntax in the script is like this:filename = ARG1 if (ARGC >= 2){tkin=ARG2} if (ARGC == 3) {set title ARG3 offset 0,-5 font ",20"}
and the calling sequence is
call 'toto.gnu' "toto.dat" 10 "TOTO"
- Using MACROS: this is a very useful, although somewhat limited,
tool; the following files are located in
$HOME/.gnuplot.d/
:put the following
png.cfg
file:fileout=ARG2 if (ARGC == 0) {print "Usage: @PNG <font_size> [<fileout>]" ; exit(0)} else {fileout=ARG2} if (ARGC < 2) {fileout="test.png"} else {fileout=ARG2} sfont=sprintf("\"Carlito,%s\"",ARG1) print sfont set term push set term pngcairo \ notransparent \ enhanced \ font @sfont \ fontscale 1 \ rounded \ crop \ size 800,800 set output fileout repl set output print "Output: ",fileout set term pop
in the general file
general.cfg
:set macro PNG = "call 'png.cfg'"
now, from
gnuplot
command line, typing the folling command will producetoto.png
file using Carlito,18 font type and size:@PNG 18 "toto.png"
Loops, iterations, etc
In the following example, we iteratively plot the weight of each species, between column 74 to 83, relative to their sum, and label each curve by the name of the species with the column number concatenated to it
plot [1e3:*] [1e-10:*] for [j=74:83] 'output/fdens_speci.out' u 2:(column(j)/(sum[k=74:83] column(k))) t columnhead(j).j
Here, we iteratively plot the weight of species taken from a list relative to the sum of all species corresponding to columns 74 to 83, and put a legend accordingly.
plot [1e3:*] [1e-10:*] for [s in "CO NH3 H2O"] 'output/fdens_speci.out' u 2:(column(s)/(sum[k=74:83] column(k))) t s
Just checking this works really:
plot [1e3:*] [1e-10:*] for [s in "CO NH3 H2O"] 'output/fdens_speci.out' u 2:(column(s)/(sum[k=74:83] column(k))) t s, \ '' u 2:($18/($75+$83+$74+$77+$96+$78)) every 10 w p t "Checking"
Here, we have overplotted the H2O species, with points, taken every 10 samples to make it easy to read. See the result below:
- A loop over three files:
- A script taken from the "Gnuplot in Action" book:
Operation on columns within a loop
s14 = "N N2 CN NO NH3 HCN" s15 = "N. N.N CN. N.O N.H3 HCN." set border 3 ratio(k)=column(word(s14,k))/column(word(s15,k)) fac(x) = x==2?2:1 # used to multiply by 2 if N2 plot for [k=1:words(s14)] 'grid_tkin_SS.out' u 3:(ratio(k)*fac(k))
Labelling
This example would put an opaque box underneath the legend, shifting leftwards the symbols towards the text by 3 character units.
set key at graph 0.1,0.95,0 left Left width -3 height 1 box opaque
The following puts thick dots at given points and label them A to C (here,
orl(x,y)
is a user-defined function):roots = "267.9 286.0 306.8" # for S = 1365 W m-2 names = "A B C" r2=252.27 r3=317.95 unset label do for [i=1:3] { x = word(roots,i) ts = 1.*x set label word(names,i) at x,orl(x,0.8) point ps 2.5 pt 7 right offset -1,1 cc=sprintf("%4s%12.1f%8.1f%12.1f", word(names,i),ts,ts-273.15,orl(x,0.8)) print cc }
Exporting plots
- pdfcairo: produces a PDF file; but not cropped… And issues with the size of the symbols: the size in the PDF does not reflect that in the window.
- export to PNG:
An alternative, which allows to control directly the DPI: export as SVG and use in-line command inkscape to export to lossless PNG:
set term svg enhanced size 500,500 set output 'test.svg' ; repl ; set output inkscape --export-area-drawing --export-png=test.png --export-dpi=300 --export-background-opacity=1 test.svg
- See alse here for some useful details.
Exporting outputs
To write the values in ASCII format:
set table "tsurf.tex" separator "&" plot [0.1:50] [10:1000] 'tsurf.dat' \ u (sprintf("%4.2f",$1)) : (sprintf("%4.2f",$2)) : (sprintf("%6.2f",ts($1,0))) with table unset table
Installation
- By default, Debian install does not include the +LIBREADLINE; use
show version long
in gnuplot to see the compilation options; - You have to install it manually…
- The following is taken from here.
- Download the source:
- You need to have the
deb-src
in the/etc/apt/sources.list
file; then
cd /usr/local/src/ sudo apt-get source gnuplot sudo apt build-dep gnuplot cd /usr/local/src/gnuplot-*.*.*/debian/
Edit the file
rules
by adding:--with-readline=gnu --enable-history-file
Then go up and compile:
cd .. sudo dpkg-buildpackage
- I noticed some loops when compiling the doc (as it seems to be) with images which are missing. Just wait…
Then, install the package:
cd .. sudo dpkg -i gnuplot-*.deb
Exclude
gnuplot
from the update:➜ src sudo apt-mark hold gnuplot gnuplot set on hold.
- You need to have the
Image/PDF files
- The basic tools are from pdftk and imagemagick
Concatenate PDF files
man pdftk pdftk A=in1.pdf B=in2.pdf cat A1-end B3-end output test.pdf
Clip an image:
convert -trim input.png output.png
Crop and clip an image: the following would split an image with four panels.
fig="input.png" ww=`identify -format "%[fx:w] %[fx:h]\n" $fig | awk '{print $1}'` hh=`identify -format "%[fx:w] %[fx:h]\n" $fig | awk '{print $2}'` mx=`expr $ww / 2` my=`expr $hh / 2` #echo $ww $mx $hh $my convert $fig -crop -$mx-$my -trim topleft.png convert $fig -crop +$mx-$my -trim topright.png convert $fig -crop -$mx+$my -trim bottomleft.png convert $fig -crop +$mx+$my -trim bottomright.png exit 0
Crop a PDF file:
pdfcrop -clip input.pdf output.pdf
Git
References
- wiki de l'ensimag
- Working with branches (and this is what you want to do!), here
are some references I found really useful:
- https://nvie.com/posts/a-successful-git-branching-model/
- https://ensiwiki.ensimag.fr/index.php?title=G%C3%A9rer_des_branches_avec_Git
- Overall scheme for project organization: nice overview
- Accordingly, a project would be organized with several branches
- master
- dev
- temporary_feature_branch (e.g. read_input_parameters)
- temporary_release_branch
- temporary_hotfix_branch
- Accordingly, a project would be organized with several branches
Basic commands
Advice: to be exectued every time you're leaving a session, no matter whether this is finished or not. The repository will thus behave as a backup. Obviously, do this in the
dev
branch, or in themaster
branch if this is version 0.x.git br # shows in which branch you are currently
- To see what has changed; you have usually two lists:
- The first lists the files registered in the repository that have been modified since the last commit.
- The second lists unregistered files
To see the changes
git diff <filename>
To make the commit:
git status git add -u <files> # list of files you will commit # git add *.f90 # would do what you think git commit -m "a message" # to commit the changes git push
Note that, if this is the first commit, then you have to do:
git push origin <new-branch-name> # to initialise the new branch with your version of the code
To revert a local file to the version in the repository:
git checkout HEAD <filename>
Create aliases
$ git config --global alias.co checkout $ git config --global alias.br branch $ git config --global alias.st status
Working with branches
- This is essential. And this is, for me, the incredible plus of git to make such complex things so easy.
Create a branch
git co -b <new_branch> # to create a new branch
To create the branch on the remote server:
$ git push origin <new_branch> Total 0 (delta 0), reused 0 (delta 0) To git+ssh://git.renater.fr:2222/scmrepos/git/chemcollapse/chemcollapse.git * [new branch] test -> test
Pushing files from a branch for the first time:
➜ chemcollapse git:(dev) ✗ git push fatal: The current branch dev has no upstream branch. To push the current branch and set the remote as upstream, use git push --set-upstream origin dev
Moving from one branch to another
git br # to see where you are git co branch_name # to move to another branch
Initiate a branch: the point to be understood is that you will create a branch starting from an existing ones. For example, creating a branch
dev
should be done like this:$ git br # check that you are in master $ git co -b dev # create the branch dev $ git br # chech that you are in dev
From this point, you are starting a dev branch from the current version of the
master
branch.Example, with a branch dedicated to improve the reading of the input parameters of ALICO:
$ git push --set-upstream origin read_input_parameters Counting objects: 22, done. Delta compression using up to 4 threads. Compressing objects: 100% (22/22), done. Writing objects: 100% (22/22), 22.47 KiB | 0 bytes/s, done. Total 22 (delta 12), reused 0 (delta 0) To ssh://git.renater.fr:2222/alico.git * [new branch] read_input_parameters -> read_input_parameters Branch read_input_parameters set up to track remote branch read_input_parameters from origin.
The option
set-upstream
tells git to push from our localread_input_parameters
branch the one with the same name in the repository. This needs to be done only for the firstpush
. After this, a simplegit push
is sufficient.To delete a local or remote branch
git branch -d the_local_branch # To delete a local branch: git push origin --delete the_remote_branch # To delete a remote branch:
Conflicts
- Resolving conflicts: see here
Problem with branches
Your branch and 'origin/master' have diverged, # and have 1 and 1 different commit(s) each, respectively.
- Taken from this thread
- check if you need to update origin.
If origin is up-to-date, then some commits have been pushed to origin from another repo while you made your own commits locally:
... o ---- o ---- A ---- B origin/master (upstream work) \ C master (your work)
- As can be seen: commit C is based on commit A because that was the latest work you had fetched from upstream at the time.
- However, before trying to push back to
origin
, someone else pushed commit B. Development history has diverged into separate paths. - You can then
merge
orrebase
. See Pro Git: Git Branching - Rebasing for details.
- Merging (not my prefered solution)
Use the
git merge
command:$ git merge origin/master
This tells Git to integrate the changes from origin/master into your work and create a merge commit. The graph of history now looks like this:
... o ---- o ---- A ---- B origin/master (upstream work) \ \ C ---- M master (your work)
- The new merge, commit M, has two parents, each representing one path of development that led to the content stored in that commit. Note that the history behind M is now non-linear.
- Rebase (my prefered solution)
Use the
git rebase
command:$ git rebase origin/master
- This tells Git to replay commit C (your work) as if you had based it on commit B instead of A. CVS and Subversion users routinely rebase their local changes on top of upstream work when they update before commit. Git just adds explicit separation between the commit and rebase steps.
The graph of history now looks like this: You can review the differences with a:
... o ---- o ---- A ---- B origin/master (upstream work) \ C' master (your work)
- Commit C' is a new commit created by the git rebase
command. It is different from C in two ways:
- It has a different history: B instead of A;
- Its content accounts for changes in both B and C; it is the same as M from the merge example.
- Note that the history behind C' is still linear which makes it my prefered choice. This approach preserves the CVS-based workflow used previously and may ease the transition. An attempt to push C' into our repository will work (assuming you have permissions and no one has pushed while you were rebasing).
The git pull command provides a shorthand way to fetch from origin and rebase local work on it:
$ git pull --rebase
This combines the above fetch and rebase steps into one command.
RENATER server
Filesystem
Mounting
basics
cd /media/username/ sudo mkdir DESTINATION sudo mount /dev/XXX DESTINATION
where the partition id
/dev/XXX
may be obtained using thelsblk
command (see here) orfdisk -l
(see also below)- General reference here
Another reference here
The correct syntax is: sudo mount -o remount,rw /partition/identifier /mount/point Where mount/point is /partition/identifier's corresponding mountpoint, as listed by the following command: mount -v | grep "^/" | awk '{print "\nPartition identifier: " $1 "\n Mountpoint: " $3}'
To identify the UUID (a unique partition identifier):
➜ ~ sudo blkid /dev/mapper/ipag--9006--vg-root: UUID="e963b1f8-17a4-44cf-8a7b-23f5eed42c67" BLOCK_SIZE="4096" TYPE="ext4" /dev/nvme0n1p3: UUID="cf67d2b6-7ea6-456a-a8df-d9bcdf504f81" TYPE="crypto_LUKS" PARTUUID="ab456709-6596-4762-a607-06641e8eea4f" /dev/nvme0n1p1: UUID="63B5-6AAC" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="d9a32a12-fea5-4d58-93a0-29b967961709" /dev/nvme0n1p2: UUID="ba818f17-62b6-4d3e-83d3-5706ea6e07c3" BLOCK_SIZE="1024" TYPE="ext2" PARTUUID="370265e2-70d1-4d37-9b20-ea9f11516156" /dev/mapper/ipag--9006--vg-swap_1: UUID="7449b68c-6993-4bf6-a687-6b5157881a74" TYPE="swap" /dev/mapper/nvme0n1p3_crypt: UUID="1oKKbs-agJx-aful-nSnK-0HF5-wzqf-UKBYRd" TYPE="LVM2_member" /dev/sda1: LABEL="phb2000" UUID="d945144b-47d2-4f42-b996-9b20b5772115" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="36a79383-28e1-446d-9fa5-a68c1c716ddf"
Set permissions of a USB drive upon mounting
➜ ~ mount /dev/sda1 on /media/hilyblap/phb2000 type ext4 (rw,nosuid,nodev,relatime,errors=remount-ro,uhelper=udisks2)
Repairing a filesystem
RAID system
- See here
How to deal with
/dev/sdb is apparently in use by the system
umount /dev/md0 mdadm --stop /dev/md0 mdadm --zero-superblock /dev/sdb # clear superblock mdadm --remove /dev/md0 # remove array
Global installation
Debian
apt
- Useful links
- update and upgrade
The following packages have been kept back
- Means that some package could not be upgraded; this could be
due to several reasons; usually, this is because of
dependency issues; most common issues are
- the new package relies on a package for which there is no update in the current distro
- the new package rests upon new package not installed in the current distro
To fix the issue, try:
sudo apt-get install the_problematic_package
- Means that some package could not be upgraded; this could be
due to several reasons; usually, this is because of
dependency issues; most common issues are
How to know which distribution is installed?
cat /etc/os-release
produces the following output:
PRETTY_NAME="Debian GNU/Linux 10 (buster)" NAME="Debian GNU/Linux" VERSION_ID="10" VERSION="10 (buster)" VERSION_CODENAME=buster ID=debian HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support" BUG_REPORT_URL="https://bugs.debian.org/"
Gnome
- DVD ripper: Handbrake
Calendar To set Monday as the first weekday ?
sudo cp /usr/share/i18n/locales/en_US /usr/share/i18n/locales/en_US.bak sudo emacs -nw /usr/share/i18n/locales/en_US
To this file, add the following line:
first_weekday 2
The logout and login.
Sound-Recorder:
- Comment definir l'entree pour enregistrement
- open sound-recorder
- open PulseAudio Volume Control (PAVC)
- start recording in sound-recorder
- set the input from the menu un the PAVC
internal –> internal microphone monitor of internal –> carte son
Bluetooth
ls /run/user/1000/gvfs/obex…
PDF viewer
- qpdfview
- View toolbar: zoomIn,scaleFactor,zoomOut,separator,continuousMode,twoPagesMode,separator,rotateLeft,rotateRight,separator,fullscreen,presentation
Softwares, other
Python
- Use
python3
instead ofpython
ADS
- Positional arguments: see this link
- Example
- pos(author:”Oort, J”,2)papers which have “J. Oort” as the second author
- pos(author:”Oort, J”,1,3)papers which have “J. Oort” as first, second, or third author
WPS
Add a dictionnary
sudo apt-get install hunspell-fr-comprehensive cd /opt/kingsoft/wps-office/office6/dicts/spellcheck sudo ln -s /usr/share/hunspell/fr_FR.aff main.aff sudo ln -s /usr/share/hunspell/fr_FR.dic main.dic cat /opt/kingsoft/wps-office/office6/dicts/spellcheck/fr_FR/dict.conf [Dictionary] DisplayName=French (Comprehensive) DisplayName[de_DE]=French (Comprehensive)
R1C1 reference style
: Menu -> Options -> General and save
Libreoffice
- To Start With a Defined Page Number:
- Click into the 1st paragraph of the page you want to set the number
- Format -> Paragraph -> Text flow.
- In the
Breaks
area, enableInsert
andWith Page Style
. Set the page number.
Gimp
- Put a white background in a transparent png
- open png
- Layer -> New layer: chose backgroud layer
- The white layer is above the png file
- Layer > Stack -> Reverse layer order
- File -> Export…
Conky
This is system monitor for X. See here.