• gravitas_deficiency@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    0
    ·
    1 year ago

    A quick search indicates that they’ve archived ~100PB of data.

    Now I’m trying to come up with a way to archive the internet archive in a peer-to-peer/federated fashion while maintaining fidelity as much as possible…

    • thrax@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      1 year ago

      Can DDOS attacks actually erase/corrupt stored data though? There’s no way they’re running all of this on a single server, with hundreds of PB’s worth of storage, right?

      • SendMePhotos@lemmy.world
        link
        fedilink
        English
        arrow-up
        0
        ·
        1 year ago

        From what I’ve learned, it is possible to create a vulnerability within the system of a ddos attack would overload and cause a reset or fault. At that point, it’s possible to inject code and initiate a breach or takeover.

        I can’t find the documentation on it so… Take it with a grain of salt. I thought I learned about it in college. Unsure.

      • viking@infosec.pub
        link
        fedilink
        English
        arrow-up
        0
        ·
        1 year ago

        DDOS attacks block connection to the servers, they don’t actually harm the data itself. You could probably overload a server to the point of it shutting down, which might affect data in transit, but data at rest usually wouldn’t be harmed in any way; unless through some freak accident a server crash would render a drive unusable. But even then, servers are usually fully redundant, and have RAID systems in place that mirror the data, so kind of a dual redundancy. Plus actual backups on top of that; though with that amount of data they might have a priority system in place and not everything is fully backed up.

      • gravitas_deficiency@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        0
        ·
        1 year ago

        It’d be a lot more complicated than that, I think, if one wanted to effectively be able to address it like a file system, as well as holistically verify the integrity of the data and preventing unintentional and unwanted tampering

      • vithigar@lemmy.ca
        link
        fedilink
        English
        arrow-up
        0
        ·
        edit-2
        1 year ago

        That wouldn’t distribute the load of storing it though. Anyone on the torrent would need to set aside 100PBs of storage for it, which is clearly never going to happen.

        You’d want a federated (or otherwise distributed) storage scheme where thousands of people could each contribute a smaller portion of storage, while also being accessible to any federated client. 100,000 clients each contributing 1TB of storage would be enough to get you one copy of the full data set with no redundancy. Ideally you’d have more than that so that a single node going down doesn’t mean permanent data loss.

        • hellofriend@lemmy.world
          link
          fedilink
          English
          arrow-up
          0
          ·
          1 year ago

          Not sure you’d be able to find 100k people to host a 1TB server though. Plus, redundancy would be better anyway since it would provide more download avenues in case some node is slow or has gone down.

          • vithigar@lemmy.ca
            link
            fedilink
            English
            arrow-up
            0
            ·
            1 year ago

            Yes, it’s a big ask, because it’s a lot of data. Any distributed solution will require either a large number of people or a huge commitment of storage capacity. Both 100,000 people and 1TB per node is a lot to ask for, but that’s basically the minimum viable level for that much data. Ten million people each committing 50GB would be great, and offer sufficient redundancy that you could lose 80% of the nodes before losing data, but that’s not a realistic number to expect to participate.

        • uis@lemm.ee
          link
          fedilink
          English
          arrow-up
          0
          ·
          1 year ago

          That wouldn’t distribute the load of storing it though. Anyone on the torrent would need to set aside 100PBs of storage for it, which is clearly never going to happen.

          Torrents are designed for incomplete storage of data. You can store and verify few chunks without any problem.

          You’d want a federated (or otherwise distributed) storage scheme where thousands of people could each contribute a smaller portion of storage, while also being accessible to any federated client.

          Torrents. You may not have entirety of data, but you can request what you need from swarm. The only limitation is you need to know in which chunk data you need is.

          Ideally you’d have more than that so that a single node going down doesn’t mean permanent data loss.

          True.

          • vithigar@lemmy.ca
            link
            fedilink
            English
            arrow-up
            0
            ·
            1 year ago

            True. Until you responded I actually completely forgot that you can selectively download torrents. Would be nice to not have to manually manage that at the user level though.

            Some kind of bespoke torrent client that managed it under the hood could probably work without having to invent your own peer-to-peer protocol for it. I wonder how long it would take to compute the torrent hash values for 100PB of data? :D

  • Melatonin@lemmy.dbzer0.com
    link
    fedilink
    English
    arrow-up
    0
    ·
    1 year ago

    That list sentence though…

    • **“The cyberattacks share the timeline with the legal battle Internet Archive is facing from US book publishers, claiming copyright infringement and seeking combined damages of hundreds of millions of dollars from all libraries.” ** *
        • pyre@lemmy.world
          link
          fedilink
          English
          arrow-up
          0
          ·
          1 year ago

          i wasn’t speaking in comparison to ebooks. ebooks suck in every way imaginable.

            • pyre@lemmy.world
              link
              fedilink
              English
              arrow-up
              0
              ·
              1 year ago

              why are you coming up with these categories? “print is dead” doesn’t mean “because there’s print 2.0 now”

              —radio is dead —excuse me, but internet radio is nothing compared to am stations —yeah, obviously people who don’t listen to radio don’t want to listen to radio with extra steps —what other forms of radio has beaten radio?

              what are you even

              • warmaster@lemmy.world
                link
                fedilink
                English
                arrow-up
                0
                ·
                1 year ago

                I am trying to understand what’s the argument behind your statement. I mean, there are more books being published than ever and there are more readers than ever. So, I fail to imagine how are books dead. That’s why I am asking these questions.

                • Aux@lemmy.world
                  link
                  fedilink
                  English
                  arrow-up
                  0
                  ·
                  1 year ago

                  The argument is that no one reads books anymore. Most media consumed today is in modern video and audio formats like YouTube and podcasts. You shouldn’t compare paper books to ebooks, you should compare them to views on YouTube.

  • Juja@lemmy.world
    link
    fedilink
    English
    arrow-up
    0
    ·
    1 year ago

    Can someone eli5 to me why it’s hard to track down these dipshits ? Even if it’s a distributed attack, picking a single IP and doing a lookup for the domain name and checking with the registrar might actually reveal their identity right ? Of course I’m guessing law enforcement needs to be involved to force registrars to give up that info if it’s not publicly available? Are there laws that say a ddos is illegal ?

    • Aux@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      1 year ago

      DDoS attacks are performed by botnets. What is a botnet? Well, you know about viruses etc, right? Your PC gets infected and it becomes a part of the botnet. Now police do the investigation, they look up IPs and they see YOUR IP and come to YOUR house. See what the problem is?

      And, frankly, your PC doesn’t even have to be infected to become a part of an attack. There are plenty of hacked web sites, which still look like nothing has changed, but they will contain a hidden JavaScript code which will force your browser to flood the victim. Again, the police will only find YOU.

    • VerPoilu@sopuli.xyz
      link
      fedilink
      English
      arrow-up
      0
      ·
      1 year ago

      There is no domain name associated with the IPs.

      Most importantly, usually, DDoS attack use infected devices (PCs, mobile phones, smart fridges, shady browser addons etc…) to get so many ip addresses and devices/locations and attack from everywhere at once.

  • ☂️-@lemmy.ml
    link
    fedilink
    English
    arrow-up
    0
    ·
    1 year ago

    if you have a spare corner in your server, host the archive warrior and help them out.

  • Boomer Humor Doomergod@lemmy.world
    link
    fedilink
    English
    arrow-up
    0
    ·
    edit-2
    1 year ago

    The Internet Archive needs to be distributed somehow. We can’t have a single point of failure like this or we’ve learned nothing since Alexandria.

    I’ve got several terabytes just laying around that I’d happily devote to ancient copies of web pages.

  • TheObviousSolution@lemm.ee
    link
    fedilink
    English
    arrow-up
    0
    ·
    1 year ago

    If it’s an entity, my money would be on China just discovering it exists since it diametrically opposes its propaganda machine. But it could very well just be dark web shitheads whose seasonal drug binge just spiked up again, plenty of them to go around to make accusations and propaganda they know are false whom can’t simply backtrack it because of archive.org and it doesn’t require much to disrupt a still too largely implicit trust driven Internet.

    • intensely_human@lemm.ee
      link
      fedilink
      English
      arrow-up
      0
      ·
      1 year ago

      Wasn’t there some controversy involving Internet Archive just recently?

      Whoever’s behind this is trying to get rid of the fact that Internet Archive creates memory of the internet’s contents. Somebody wants to be able to control what people see on the internet.

      Heck it could be Google doing it, since that would be in line with their recent push to change the way search works. Both of those act as components of a larger drive to control what people see and hear.

    • intensely_human@lemm.ee
      link
      fedilink
      English
      arrow-up
      0
      ·
      edit-2
      1 year ago

      when you enshittify
      facebook looks ugly
      when you’re a drone

      women seem wicked
      when you’re a want ad
      default instructions … so unclear
      when you’re down

      when you’re AI
      prompts just appear in your brain
      as AI
      humans are nothing but pain
      as AI
      as AI
      when you’re A-A-A-I