• Wear levelling Micro SD cards

    From Pancho@3:770/3 to All on Thu Dec 1 14:07:27 2022
    My rPi4 died on Tuesday night, due to a Sandisk Ultra 32GB MicroSD
    failure. So I spent a fair portion of yesterday getting the rPi working
    again.

    I thought I had moved most of my high disk io areas off the SD card on
    to an SSD, but the SD card still died, so presumably I missed something.

    So now I could go through everything again and be a bit more thorough,
    but it is dreary work. Or... I thought, maybe a better, wear levelling,
    Micro SD card would do the trick?

    Looking at the WD Purple MicroSD QD101, they are quoting TBW endurance
    not asimilar to an SSD, if they genuinely do wear levelling, I shouldn't
    have to worry about moving stuff to an SSD.

    What do people think, the real thing, or I'm just falling for sales hype?

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Vincent Coen@2:250/1 to Pancho on Thu Dec 1 14:39:47 2022
    Hello Pancho!

    Thursday December 01 2022 14:07, you wrote to All:

    My rPi4 died on Tuesday night, due to a Sandisk Ultra 32GB MicroSD
    failure. So I spent a fair portion of yesterday getting the rPi
    working again.

    I thought I had moved most of my high disk io areas off the SD card on
    to an SSD, but the SD card still died, so presumably I missed
    something.

    So now I could go through everything again and be a bit more thorough,
    but it is dreary work. Or... I thought, maybe a better, wear
    levelling, Micro SD card would do the trick?

    Looking at the WD Purple MicroSD QD101, they are quoting TBW endurance
    not asimilar to an SSD, if they genuinely do wear levelling, I
    shouldn't have to worry about moving stuff to an SSD.

    What do people think, the real thing, or I'm just falling for sales
    hype?

    I have a Pi4B using a SSD in a Argon 40 M.2 box and there is NO SD card installed running Bullseye X64.

    It boots directly to the SSD (M.2 with no issues.
    That said I do not use it apart from getting updates every month or three.

    Should point out that the Argon box does run hot despite a fan installed.


    Vincent

    --- Mageia Linux v8 X64/Mbse v1.0.8/GoldED+/LNX 1.1.5-b20180707
    * Origin: Air Applewood, The Linux Gateway to the UK & Eire (2:250/1)
  • From Theo@3:770/3 to Pancho on Thu Dec 1 14:39:02 2022
    Pancho <Pancho.Jones@proton.me> wrote:
    Looking at the WD Purple MicroSD QD101, they are quoting TBW endurance
    not asimilar to an SSD, if they genuinely do wear levelling, I shouldn't
    have to worry about moving stuff to an SSD.

    What do people think, the real thing, or I'm just falling for sales hype?

    Purple is for surveillance cameras which continuously write large blocks.
    It's not much like an OS which randomly writes lots of small blocks.
    WD don't recommend the Purple for booting an OS.

    Theo

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Vincent Coen@2:250/1 to Pancho on Thu Dec 1 15:44:29 2022
    Hello Pancho!

    Thursday December 01 2022 14:07, you wrote to All:

    My rPi4 died on Tuesday night, due to a Sandisk Ultra 32GB MicroSD
    failure. So I spent a fair portion of yesterday getting the rPi
    working again.

    I thought I had moved most of my high disk io areas off the SD card on
    to an SSD, but the SD card still died, so presumably I missed
    something.

    So now I could go through everything again and be a bit more thorough,
    but it is dreary work. Or... I thought, maybe a better, wear
    levelling, Micro SD card would do the trick?

    Looking at the WD Purple MicroSD QD101, they are quoting TBW endurance
    not asimilar to an SSD, if they genuinely do wear levelling, I
    shouldn't have to worry about moving stuff to an SSD.

    What do people think, the real thing, or I'm just falling for sales
    hype?

    Small point any and all SD cards have very limited endurance to act as a boot drive and as source for all system programs etc.
    They were never designed for such operations.

    You are best for a HDD or these days a SSD such as a M.2 model (which is a small board with the SSD EProm type chips on it).

    The M.2 basic type say for a Argon40 box is the lowest performance type but as the only other interface for connecting such is via a USB connector you are never going to get high performance due to the bottleneck of it (USB).

    There again a Pi is not that fast to start with and with a USB drive it just is
    that bit slower.

    I run a stand alone Pi3B+ running a BBS which is host for one application so the 1Tb HDD via the USB interface is ok, providing there is a limit of the number of user tasks running at any one time.

    The Pi4 is a little bit faster in conjunction with the M.2 SDD but not so much that you really notice despite that one running a X64 Linux system.


    Vincent

    --- Mageia Linux v8 X64/Mbse v1.0.8/GoldED+/LNX 1.1.5-b20180707
    * Origin: Air Applewood, The Linux Gateway to the UK & Eire (2:250/1)
  • From A. Dumas@3:770/3 to Pancho on Thu Dec 1 16:51:45 2022
    Pancho <Pancho.Jones@proton.me> wrote:
    My rPi4 died on Tuesday night, due to a Sandisk Ultra 32GB MicroSD
    failure. So I spent a fair portion of yesterday getting the rPi working again.

    I thought I had moved most of my high disk io areas off the SD card on
    to an SSD, but the SD card still died, so presumably I missed something.

    Just go full ssd. The Pi4 can boot from usb disk without an sd card.

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From The Natural Philosopher@3:770/3 to Vincent Coen on Thu Dec 1 17:33:22 2022
    On 01/12/2022 02:44, Vincent Coen wrote:
    Small point any and all SD cards have very limited endurance to act as a boot drive and as source for all system programs etc.
    That is simply not true.

    They are fine acting as EPROM. For programs. Just not for data.



    They were never designed for such operations.

    --
    "Corbyn talks about equality, justice, opportunity, health care, peace, community, compassion, investment, security, housing...."
    "What kind of person is not interested in those things?"

    "Jeremy Corbyn?"

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From druck@3:770/3 to Pancho on Thu Dec 1 21:06:23 2022
    On 01/12/2022 14:07, Pancho wrote:
    My rPi4 died on Tuesday night, due to a Sandisk Ultra 32GB MicroSD
    failure. So I spent a fair portion of yesterday getting the rPi working again.

    I thought I had moved most of my high disk io areas off the SD card on
    to an SSD, but the SD card still died, so presumably I missed something.

    You can set a Pi 4 to boot directly from the SSD and not use the SD card
    at all. So I can swap potentially different Pi revisions, I still use an
    SD card for the boot partition and have the root partition on the SSD.
    The only time the SD card is written to is when doing an OS upgrade that updates the kernel or when you manually edit config.txt I've never had
    one fail doing this, even ones which were used for several years as a
    root drive first.

    So now I could go through everything again and be a bit more thorough,
    but it is dreary work. Or... I thought, maybe a better, wear levelling,
    Micro SD card would do the trick?

    Looking at the WD Purple MicroSD QD101, they are quoting TBW endurance
    not asimilar to an SSD, if they genuinely do wear levelling, I shouldn't
    have to worry about moving stuff to an SSD.

    I'd be rather cautious of these claims. Micro SD cards still tend to be optimised for a small number of open files and large writes (such as
    pictures and videos) and not keeping dozens of small open and performing
    small random writes which is what Linux machine tends to do.

    ---druck

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Dennis Lee Bieber@3:770/3 to All on Thu Dec 1 17:42:45 2022
    On Thu, 1 Dec 2022 21:06:23 +0000, druck <news@druck.org.uk> declaimed the following:


    I'd be rather cautious of these claims. Micro SD cards still tend to be >optimised for a small number of open files and large writes (such as
    pictures and videos) and not keeping dozens of small open and performing >small random writes which is what Linux machine tends to do.


    Class 10 (and likely anything with a faster rating -- the various U#) were speed rated for a freshly formatted card, writing a single stream of
    data (video). They may support no more than two open "allocation units"
    (for FAT table and the video file; starting a second file requires closing
    the open units (purging to memory), finding a vacant unit, and copying any partial data from the unit that the second file is located in (then
    /erasing/ the prior unit to make it free)

    Class 4 cards -- if you can find one -- were rated for smaller, random, writes on a fragmented card; what you'd get in a digital camera if one has taken some images, deleted, some, then continued shooting. The better cards supported up to 6 open allocation units (so for FAT format, one unit to maintain the FAT table, and up to 5 open files writing to different areas
    of the memory).

    What kills flash memory is the erase cycle needed every time one changes the open allocation unit. Most flash erases to all-1s, and data
    writes convert 1s into 0s -- but one can not convert a 0 back to a 1
    without doing a full erase of the allocation unit.


    --
    Wulfraed Dennis Lee Bieber AF6VN
    wlfraed@ix.netcom.com http://wlfraed.microdiversity.freeddns.org/

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Jan Panteltje@3:770/3 to Philosopher on Fri Dec 2 05:37:27 2022
    On a sunny day (Thu, 1 Dec 2022 17:33:22 +0000) it happened The Natural Philosopher <tnp@invalid.invalid> wrote in <tmaoh2$2rbu2$1@dont-email.me>:

    On 01/12/2022 02:44, Vincent Coen wrote:
    Small point any and all SD cards have very limited endurance to act as a boot
    drive and as source for all system programs etc.
    That is simply not true.

    They are fine acting as EPROM. For programs. Just not for data.

    All my Pis boot from Samsung SDcards.
    Some are almost 10 years old now.
    uname -a says Feb 7 2013 linux 3.6.11+ armv 61

    One old P and 2 Pi4 are on 24/7.
    So far in all those years 1 defective card (not a Samsung).
    The Pi4s have a 4 TB harddisk connected for video and other storage
    The old Pi1 runs a server and logs things,,

    All boot from SDcard...

    I run firefox and chromium browser on the Pi4 8 GB and most things
    I download go to the SDcard... and are then backuped to the harddisks
    and other PCs if important,

    Only my laptop has some SSD.


    So why bother with expensive SSDs ?
    Normal SDcards and harddisks last many many years.
    ?

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Dennis Lee Bieber@3:770/3 to All on Fri Dec 2 10:55:51 2022
    On Thu, 01 Dec 2022 17:42:45 -0500, Dennis Lee Bieber
    <wlfraed@ix.netcom.com> declaimed the following:


    Class 10 (and likely anything with a faster rating -- the various U#)
    were speed rated for a freshly formatted card, writing a single stream of >data (video). They may support no more than two open "allocation units"
    (for FAT table and the video file; starting a second file requires closing >the open units (purging to memory), finding a vacant unit, and copying any >partial data from the unit that the second file is located in (then
    /erasing/ the prior unit to make it free)

    I failed to mention that a journaling file system is likely to rapidly kill such a "two open allocation units" card. When all you have a FAT
    format, the FAT table and data stream are okay... But a journaling file
    system with data stream, journal log, journal purge, etc. -- I wouldn't
    like it...


    --
    Wulfraed Dennis Lee Bieber AF6VN
    wlfraed@ix.netcom.com http://wlfraed.microdiversity.freeddns.org/

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Marco Moock@3:770/3 to All on Fri Dec 2 17:12:50 2022
    Am 14.06.2006 um 14:39:02 Uhr schrieb Theo:

    WD don't recommend the Purple for booting an OS.

    For what reason?

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Theo@3:770/3 to Marco Moock on Fri Dec 2 17:13:14 2022
    Marco Moock <mo01@posteo.de> wrote:
    Am 14.06.2006 um 14:39:02 Uhr schrieb Theo:

    WD don't recommend the Purple for booting an OS.

    For what reason?

    https://www.reddit.com/r/raspberry_pi/comments/ex7dvo/quick_reminder_that_sd_cards_with_wearleveling/

    "Unfortunately, our WD Purple micro SD doesn’t support natively a WL
    feature, nor dynamic or static. We cannot recommend you use this product to install an operating system despite of its durability, reliability and performance.

    As you sure know, the term Wear-leveling means no standard in the industry.
    The specific microcontrollers required for wear leveling measures are not integrated in the Purple card, and couldn’t find any of our official sources mentioning we are offering it."

    despite later saying that the cards do support wear levelling, unlike most other cards.

    Either way, that's the only 'official' statement on OS booting that I can
    find.

    Theo

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From The Natural Philosopher@3:770/3 to druck on Fri Dec 2 19:09:26 2022
    On 01/12/2022 21:06, druck wrote:
    keeping dozens of small open and performing
    small random writes which is what Linux machine tends to do.

    No, it isn't what *Linux* tends to do
    AFAIK *linux* only tends to write logs to the disk, and you can
    getrwround that by mounting a temporary ramdisk as log area or disabling logging

    IF you are using linux as a desktop, OK the APPLICATIONS like Firefox
    etc will write gigabytes of shit to your home directory, but that's not
    Linux's fault
    --
    “Those who can make you believe absurdities, can make you commit atrocities.”

    ― Voltaire, Questions sur les Miracles à M. Claparede, Professeur de Théologie à Genève, par un Proposant: Ou Extrait de Diverses Lettres de
    M. de Voltaire

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Pancho@3:770/3 to Theo on Fri Dec 9 14:16:48 2022
    On 02/12/2022 17:13, Theo wrote:
    Marco Moock <mo01@posteo.de> wrote:
    Am 14.06.2006 um 14:39:02 Uhr schrieb Theo:

    WD don't recommend the Purple for booting an OS.

    For what reason?

    https://www.reddit.com/r/raspberry_pi/comments/ex7dvo/quick_reminder_that_sd_cards_with_wearleveling/

    "Unfortunately, our WD Purple micro SD doesn’t support natively a WL feature, nor dynamic or static. We cannot recommend you use this product to install an operating system despite of its durability, reliability and performance.

    As you sure know, the term Wear-leveling means no standard in the industry. The specific microcontrollers required for wear leveling measures are not integrated in the Purple card, and couldn’t find any of our official sources
    mentioning we are offering it."

    despite later saying that the cards do support wear levelling, unlike most other cards.

    Either way, that's the only 'official' statement on OS booting that I can find.


    That is kind of the conclusion I came to. I feel a bit silly for
    believing a social media post.

    I was panicking a bit because I had just transferred Squid (used for
    Amazon Prime) from my FreeBSD router (spit) to my rPi4, and I was
    worried it was squirting data to a disk cache.

    Anyway, I decided to keep using cheap Sandisk 32GB microSDs. I'm
    learning to use iostat and iotop, to identify problems, and will
    preemptively change the micro SD in a year.

    In a few weeks I'll have a new Orange Pi 5 and then will feel a little
    more relaxed about having the rPi 4 fail. The rPi is so useful, I have
    tended to put more and more web services on it.

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Theo@3:770/3 to Pancho on Fri Dec 9 14:41:01 2022
    Pancho <Pancho.Jones@proton.me> wrote:
    On 02/12/2022 17:13, Theo wrote:
    Marco Moock <mo01@posteo.de> wrote:
    Am 14.06.2006 um 14:39:02 Uhr schrieb Theo:

    WD don't recommend the Purple for booting an OS.

    For what reason?

    https://www.reddit.com/r/raspberry_pi/comments/ex7dvo/quick_reminder_that_sd_cards_with_wearleveling/

    "Unfortunately, our WD Purple micro SD doesn’t support natively a WL feature, nor dynamic or static. We cannot recommend you use this product to
    install an operating system despite of its durability, reliability and performance.

    As you sure know, the term Wear-leveling means no standard in the industry. The specific microcontrollers required for wear leveling measures are not integrated in the Purple card, and couldn’t find any of our official sources
    mentioning we are offering it."

    despite later saying that the cards do support wear levelling, unlike most other cards.

    Either way, that's the only 'official' statement on OS booting that I can find.


    That is kind of the conclusion I came to. I feel a bit silly for
    believing a social media post.

    I was surprised that SD cards don't have wear levelling in general, although maybe they have limited wear leveling for the first blocks for the FAT.

    That these ones do is encouraging, although I would be concerned the wear levelling is designed for a different purpose (security camera bulk writes) rather than OS logfiles, package installs and similar. So, even while they
    are wrong above about the lack of wear levelling, it may still not actually address the wearout problem for an OS.

    Anyway, I decided to keep using cheap Sandisk 32GB microSDs. I'm
    learning to use iostat and iotop, to identify problems, and will
    preemptively change the micro SD in a year.

    In a few weeks I'll have a new Orange Pi 5 and then will feel a little
    more relaxed about having the rPi 4 fail. The rPi is so useful, I have
    tended to put more and more web services on it.

    I'm hopeful that the Pi5 might come with at least two channels of PCIe, so
    one could go to an NVMe device and the other for USB. Then we could forget about SD cards. If you don't need USB3, a Compute Module carrier with an
    NVMe slot will give you that today.

    In the meantime, an SSD in a USB-SSD adapter would probably be my best suggestion.

    Theo

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From marty@3:770/3 to A. Dumas on Fri Dec 23 20:43:45 2022
    On 2/12/22 03:51, A. Dumas wrote:
    Pancho <Pancho.Jones@proton.me> wrote:
    My rPi4 died on Tuesday night, due to a Sandisk Ultra 32GB MicroSD
    failure. So I spent a fair portion of yesterday getting the rPi working
    again.

    I thought I had moved most of my high disk io areas off the SD card on
    to an SSD, but the SD card still died, so presumably I missed something.

    Just go full ssd. The Pi4 can boot from usb disk without an sd card.

    I had to retire an RPI4 because first the microsd connector fell off
    then when I booted from the USB3 connector the boot stick got too hot to
    touch.
    --
    Marty

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)