Tag Archives: Opensuse

Opensuse Leap, dolphin and abysmal slow data transfer between two external USB 3 disks

Posted on by

Today I had to copy a set of multiple directories, each with around 600 sub-directories and a total of 420,000 singular files, between two external USB-3 hard disks. Both the USB controllers of the mainboard and the controllers of the external disks were USB-3 capable. My PC has 64 GB RAM. Which in this case was of no help. I got the problem on an Opensuse Leap 15.3-system, but you may find it on other Linux-OS, too.

The problem

First I started to copy with the help of KDE’s Dolphin. The transfer rates were really bad – something about 5 MB/s. Would have had to wait for hours.

Then I tried with just a standard cp-command. The rates rose to something like 20 MB/s to 25 MB/s. But this was not a reasonable transfer rate either.

A workaround

A search on the Internet then gave me the following two commands

mytux:~ # echo $((48*1024*1024)) > /proc/sys/vm/dirty_bytes
mytux:~ # echo $((16*1024*1024)) > /proc/sys/vm/dirty_background_bytes

Note that setting values on these system parameters automatically sets the standard values of 

mytux:~ # cat /proc/sys/vm/dirty_ratio
20
mytux:~ # cat /proc/sys/vm/dirty_background_ratio
10

to zero! And vice versa. You either have to use dirty_bytes and dirty_background_bytes OR the related ratio-parameters!

The effect of the above settings was remarkable: The total transfer rate (i.e. reading from one disk and writing to another; both on the same controller) rose to 130 MB/s. Still far away from theoretical rates. But something I could live with.
It would be worth to experiment more with the size of the parameters, but I was satisfied regarding my specific problem.

Conclusion

An efficient data transfer on Linux systems between two external USB-3 disks may require system parameter settings by the user.
I admit: The whole thing left me baffled. For discussions and hints please refer to the Internet. It seems that the whole performance reduction is worst for users who have a lot of RAM. And the problem exists since 2014. Unbelievable!

Links

https://gist.github.com/ 2E0PGS/ f63544f8abe69acc5caaa54f56efe52f
https://www.suse.com/ support/kb/doc/? id=000017857
https://archived.forum.manjaro.org/ t/decrease-dirty-bytes-for-more-reliable-usb-transfer/62513
https://unix.stackexchange.com/ questions/ 567698/slow-transfer-write-speed-to-usb-3-flash-what-are-all-possible-solutions
https://unix.stackexchange.com/ questions/ 23003/ slow-performance-when-copying-files-to-and-from-usb-devices
https://lonesysadmin.net/ 2013/12/22/ better-linux-disk-caching-performance-vm-dirty_ratio/
https://forum.manjaro.org/t/the-pernicious-usb-stick-stall-problem/52297
https://stackoverflow.com/ questions/ 6834929/linux-virtual-memory-parameters

 

Replacing unstable Blender 2.82 on Leap 15.3 with flatpak or snap based Blender 3.1

Posted on by

I wanted to work a bit more with my S-curve project in Blender. See e.g.

Blender – complexity inside spherical and concave cylindrical mirrors – IV – reflective images of a Blender variant of Mr Kapoor’s S-curve

At my present location I have to use my old but beloved laptop. As a new start in the project I just wanted to make a little movie showing dynamic changes in the reflections of some moving spheres in front of the S-curve’s metallic surface. As I had already created a few Blender movies some years ago I expected this to be an easy job. But I ran into severe trouble. A major reason was the fact that Opensuse does not provide a reasonably usable RPM of a current Blender versions for Leap 15.3.

I had upgraded the laptop to Leap 15.3 in December, 2022. The laptop has an Optimus system. Of course, I prefer the Nvidia card when creating scenes with Blender. I had had some minor problems with prime-select before. But after recent updates I did not get the Nvidia card running at all – neither with Bumblebee nor with Opensuse’s prime-select. I had to invest an hour to solve this problem. After a complete uninstall of Bumblebee and bbswitch and a de- and re-installation of the Nvidia drivers and suse-prime I got prime-select working in the end.

Just to find that Blender version 2.82 provided by the Leap 15.3 OSS repository was not really stable in a Leap 15.3 environment.

Blender 2.82 unstable on Opensuse Leap 15.3

I tried to create a movie with 100 frames (400×200) in Matroska format with H.264 encoding. Rendering of the complex S-curve reflections with a strongly reduced number of light ray samples still took considerable time with the Cycles renderer on 2 out of 4 real (and 8 hyperthreaded) CPU cores. It happened that the movie – after rendering an hour – was not baked correctly. In some runs Blender crashed after having rendered around 40 frames.

Therefore I changed my policy to rendering just a collection of PNG images. The risk of wasting something like 8 hours of CPU time for a real animation was just too big. But, a new disaster happened when I wanted to open a “video editing”- project in Blender to stitch the images together to create the final movie. Blender just crashed. Actually, Blender crashed on Leap 15.3 whenever I tried to open any project type other than “General”. Not funny … I had not experienced something like this on Leap 15.2.

Unfortunately, and in contrast to Tumbleweed there is no official and recent version of Blender available for Leap 15.3. So much about stability and reliability of Leap. What to do?

Canonical’s snap service or Flatpak to the rescue

I had read about SNAP of Canonical before – but never tried it. Canonical’s strange side steps during the evolution of Linux have always been disturbing for me. Regarding “snap” I also had a newspaper article in mind about a controversy with Linux Mint – with claims that snap has closed proprietary components and is transmitting telemetry data to Canonical. Working with root rights …

Therefore, a flatpack installation had to be considered, too…. Risks are small on my old laptop. I just tried – despite concerns and doubts regarding snap.

Blender 3.1 via “snap“

I just followed the instructions for Opensuse on https://snapcraft.io/docs/installing-snap-on-opensuse to install “snap” on my Leap 15.3 system. As root: 

zypper addrepo --refresh     https://download.opensuse.org/repositories/system:/snappy/openSUSE_Leap_15.3     snappy
zypper install snapd
systemctl enable --now snapd
systemctl enable --now snapd.apparmor
rcsnapd status
# install blender
snap install blender --channel=3.1/stable --classic

Another description can be found here:
https://www.linuxcapable.com/how-to-install-snap-snap-store-snapcraft-on-opensuse-leap-15/

The run-time environment of snap is handled by a daemon “snapd”. We find the majority of the installed files in four directories

/snap           -- 1.1 GB thereof 686 MB for Blender
/usr/lib/snap   -- 48 MB
/var/snap       -- almost nothing 
/var/lib/snapd  -- 360 MB

In the first directory you also find the installed applications. In my case “Blender 3.1.2”.
The folder “/snap” contains around 1.1 GB. Blender is part of it and consumes 686 MB.

The startup of Blender via 

/snap/bin/blender &

takes about 2 secs. Loading my S-curve test file took even longer. Otherwise snap’s Blender version 3.1.2 seemed to work perfectly. All the bugs I had seen with 2.82 were and are gone. And: I could not really notice a difference in performance when working with 3D objects in the Blender viewport. For a render test case I found 17.72 secs per frame on average. Memory release after leaving Blender seemed to be OK.

Blender 3.1 via Flatpak

Then I tried a Blender installation with flatpak. The installation is almost as simple as for snap. See https://www.flatpak.org/ setup/ openSUSE. Which sums up more or less to issuing the following commands : 

zypper install flatpak
flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo
flatpak install flathub org.blender.Blender

The required packages of the Opensuse repository were

Regarding the installation I was surprised to find that flatpak requires much more space than snap. The majority of flatpack files is located in the following folder: 

/var/lib/flatpak  -- 2.3 GB, thereof around 628 MByte for Blender

So, in comparison to snap, this makes a remarkable huge difference regarding the required files to provide a working runtime environment! I understand that for just one application many files have to be provided for a stable run-time environment on KDE – but in comparison to the snap installation it appears bloated.

We start flatpak’s Blender via 

flatpak run org.blender.Blender &

The startup time was comparable to the snap installation – no difference felt. The test render time per frame was 18.95 secs. So that was 1.2 secs or 6.5 % slower than snap. Reason unclear. The difference after repeated trials was sometimes a bit smaller (4%), but sometimes also bigger (8.5%) -depending on the start order of the blender applications. But on average snap’s Blender was always a bit faster. I regard the difference is not really as problematic. But it is interesting. Memory release worked also perfectly with flatpak’s Blender installation.

Conclusion

Before you start struggling with the Blender 2.82 binary Opensuse provides, it is really worth trying a flatpak-based or a snap-based installation of Blender 3.1.2. On my system both worked perfectly.

Before you make a decision whether to go with flatpak or snap take into account the critical points which were discussed on the Internet regarding proprietary aspects of snap. Personally, I am going to use flatpak for Blender.
Otherwise, my opinion is that distributors like Opensuse should provide an important application as Blender in its present version as a binary and resolve dependencies as soon as possible. For me a flatpack installation is just a compromise. And it cost me a lot of valuable SSD space.

Links

https://www.linux-community.de/ausgaben/ linuxuser/2018/02/dreikampf/
https://hackaday.com/2020/06/24/whats-the-deal-with-snap-packages/
https://www.makeuseof.com/snap-vs-appimage-vs-flatpak/

Ceterum censeo: The worst living fascist and war criminal today who must be isolated, denazified and imprisoned is the Putler.

 

Switching between Intel and Nvidia graphics cards on an Optimus laptop with Leap 15.3

Posted on by

At my present location an old laptop with an Optimus system still provides me with good services. Recently, I installed Opensuse Leap 15.3 on it. Since then I have tried almost all I can think of to get Bumblebee running smoothly on it. In the end I gave up – sometimes “primusrun” worked, sometimes not – especially with more complex applications like Blender. And even if “primusrun” worked – “optirun” most often did not.

So, I had a look at “prime-select” installed by the packet “suse-prime” of Opensuse. Again I had mixed experiences: This worked sometimes from the command line and sometimes not. There were multiple things which were somehow interwoven:

  • bbswitch intervened or was used by prime-select when switching to Intel – and when it did and turned the nvidia card off, you could activate it again, but afterward the Nvidia device it was not recognized by the native Nvida driver or by prime-select. Most often I had to reboot.<
  • When I was lucky and after reboots got a configuration where the Nvidia driver was loaded in parallel to the i915 for Intel and used “prime-select” together with an init 3 / init 5 sequence the display manager sddm did not come up. Others had this problem, too.
  • To switch between the graphics cards it was absolutely necessary to login into my graphical KDE desktop, use a root terminal and switch to the Nvidia card with prime-select from there. Then I had to log out and if I was lucky the display manager came up and the Nvidia card was really active.

But do not ask me, what I had to do in which order to get the aspired result. All in all the behavior of “prime-select” was a bit unpredictable. Then after updating a lot of packets yesterday I could no longer get prime-select to do the right things any more. Most often I got the message:

ERROR: Unable to query GPU information

Or : No such device found.

Solution Part 1: Deinstallation of unrequired RPMs and reinstallation of required RPMs

In the end I deinstalled Bumblebee (as recommended by posts in Suse forums) and also bbswitch. Afterward I reinstalled the packets “suse-prime” and “plasma5-applet-suse-prime” from the Leap 15.3 Update-repository (i.e. the Update repo for the Enterprise system) and the Nvidia-drivers from Opensuse’s Nvidia repository. I rebooted and my “sddm” display-manager came up – with the Intel driver active – but the Nvidia driver was loaded already nevertheless. Checked with lsmod.

Later, I found that all files in “/etc/prime” had been rewritten. I suspect that I had something in there which was no longer compatible with the latest drivers and prime-versions. In addition the file “90-nvidia.conf” has been replaced in “/etc/X11/xorg.conf.d“.

Solution Part 2: Use the “Suse Prime Selector” applet!

Then I used the applet in the KDE Plasma desktop to switch to Nvidia. This seemed to work and led to no errors. I just had to log out of my KDE Plasma session and in again to work with KDE on a running Nvidia card. (Ignore the message that you have to reboot).
Afterward I also tested switching by using the command “prime-select” as root in a terminal window of the KDE desktop. Worked perfectly, too.

Conclusion

I am not sure what the ultimate reason was to get prime-select running. The deinstallation of Bumblebee and bbswitch? The new Prime and Nvidia configuration files? Or a new udev during other updates?

Anyway – using the Suse Prime Selector applet or just using the command “prime-select” as root in a terminal window inside KDE Plasma has become a very stable way on my Leap 15.3 laptop with KDE to switch between graphics cards reliably. I still have to log in and log out again of the KDE session – but so far I have never experienced any problems with the display-manager again.

Just for information: I am using the following Nvidia drivers packages:

The prime-select packet version is:

Links

https://www.reddit.com/ r/ openSUSE/ comments/ ib9buv/ opensuse_kde_suse_ prime_selector_ applet_for/
https://forums.developer.nvidia.com/ t/ kubuntu-18-04-gives-black-screen-when-sddm-starts-no-login/ 61034/11
https://wiki.archlinux.org/ title/ PRIME# Configure_applications_%20 to_render_using_GPU
https://www.opensuse-forum.de/ thread/ 63871-suse-prime-error-unable-to-query-gpu-information/
https://opensuse.github.io/ openSUSE-docs-revamped-temp/hybrid_graphics/

Ceterum censeo: The worst living fascist and war criminal, who must be isolated, denazified and imprisoned, is the Putler.