3

I am running a linux system with an app that constantly reads/writes many small files (e.g close to zero bytes) to a point where 1000-4000 files read/written each minute causing disk innodes close to 100%.

What is the best HDD setup to improve performance when these files are read/write by more instances at once?

  • 4 SSD in raid 0?
  • 2/3 SSD in raid 0?
  • 4 HDD raid zero? (because I think SSDs drop in performance after so many permanent writes)
  • raid 0 included in Asus motherboard?
  • dedicated raid car?
Improve this question
2
  • If this is a server type system, have you thought about using a RAM disk? If you are concerned with actually retrieving the data at a later time, you can have the files written out of RAM back to disk. Some RAM disk applications come with a function that allows the contents of the RAM disk to be written to file and reloaded on startup.
    – Bennett Yeo
    Commented Jun 14, 2018 at 8:25
  • 1
    What platform are you on currently? What's your budget like? How big are the files all combined? My thought is potentially some sort of 3D Xpoint solution, but it may or may not be feasible depending on what you're doing.
    – JMY1000 ♦
    Commented Jun 14, 2018 at 9:14

1 Answer 1

Reset to default
1

In general, performance RAID and SSD are not a thing you need to do. The performance improvement is pretty minimal, and you'd only want to use RAID as a way to maintain multiple drives in case one begins to fail, allowing the other drives in the cluster to handle the load. If you're talking 10+ drives, performance RAID can come back into play.

HDD simply wont improve your performance compared to SSD. An SSD will r/w faster than 3 HDD in RAID 0. You can't beat the abandonment of mechanical parts in an SSD.

I recommend using a calculator like this one to get an idea of your I/O speed results vs. requirements. Dedicated RAID vs onboard RAID is pretty much identical as far as performance and reliability.

In all cases, SSDs do indeed drop performance over time, but with modern SSDs, TRIM-style operations prolong this to a point that it won't slow down to anything close to HDD speeds until the drive is on its last legs, which is roughly two years after an HDD would have died anyway.

If capacity or cost are less of an issue, and all you want is performance, then a RAM drive may be the way to go. These are faster than SSD and not subject to the same wear, and if you're doing that much work at a time, this might be a better approach. You'd still want periodic backup to permanent media, but RAM drives are pretty much designed for how you described your use-case.

Improve this answer
2
  • RAM Drives (i.e. using RAM as a pretend drive) will offer the best performance, with access times measured in nano-seconds (rather than milliseconds). However, they are volatile - so a power cut or PC failure will me you will lose data. How much depends of the RAM disk s/w used and how it has been configured to work. A UPS can partially mitigate this, but unless this application can handle the occasional, abrupt loss of data, this is not the best approach.
    – CJM
    Commented Jun 19, 2018 at 16:31
  • I wouldn't recommend a RAM drive unless you pair it with periodic writes to less volatile memory, such as a task which backs up the contents to an HDD or SSD as a "checkpoint" within acceptable tolerance.
    – CDove
    Commented Jun 20, 2018 at 11:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.