The capacity and speed curves of drives is quite different. When the first TB drives were released about 15 years ago, the fastest ones were about 100MB/s. Now you can get 20TB drives (20x capacity) and the fastest ones are about 300MB/s (3x speed). It is just far easier and cheaper to make a drive twice as big than it is to make it twice as fast.
SSDs are a different animal, but have some similar characteristics. Within the same generation (e.g. m.2 pcie gen 4), you can get drives that have a lot more capacity but have roughly the same access speeds (i.e. the 2TB version is very similar to the 1TB version). The speed increases between generations is much better than with HDD. Pcie gen 3 drives seemed to max out about 3500MB/s while the gen 4 drives are about double that. I have seen reports that gen 5 drives might double it again
With HAMR technology we might get HDD drives with capacities in the 50TB-100TB range. You can bet that the speed won't be 5x current technology even if they get dual-actuators in them. There will need to be some kind of breakthrough technology to improve it significantly.
This is why we need better data management systems. If the meta-data (e.g. file table) is only 1% of the data that is still a lot of data to read in and store in RAM. We need better systems where the file records are much smaller.
https://didgets.substack.com/p/where-did-i-put-that-file
SSDs are a different animal, but have some similar characteristics. Within the same generation (e.g. m.2 pcie gen 4), you can get drives that have a lot more capacity but have roughly the same access speeds (i.e. the 2TB version is very similar to the 1TB version). The speed increases between generations is much better than with HDD. Pcie gen 3 drives seemed to max out about 3500MB/s while the gen 4 drives are about double that. I have seen reports that gen 5 drives might double it again
With HAMR technology we might get HDD drives with capacities in the 50TB-100TB range. You can bet that the speed won't be 5x current technology even if they get dual-actuators in them. There will need to be some kind of breakthrough technology to improve it significantly.
This is why we need better data management systems. If the meta-data (e.g. file table) is only 1% of the data that is still a lot of data to read in and store in RAM. We need better systems where the file records are much smaller. https://didgets.substack.com/p/where-did-i-put-that-file