MiniLake Is a Local Databricks API Emulator with SQL, Delta Lake and Spark

 (dmux.github.io)

40 points | by kermatt 1 day ago

5 comments

  • gavinray 51 minutes ago
    If this actually works, this is such a huge boon. Let me share why:

    I work on connectors to various datasources, mostly SQL DB's

    The most useful tests are the integration tests, sending actual queries to the source

    For most DB's we have a giant Docker Compose which runs a local instance

    But things like Snowflake, Databricks, Redshift, Athena suck to test because we need an external hosted instance

    I would kill to be able to run test versions of all of these locally

    [-]
  • akbarian 53 minutes ago
    Getting tired of the standard AI design, all websites look the same now
    [-]
    • pjmlp 41 minutes ago
      Before it was all those common template sites.
  • ngruhn 2 hours ago
    You should call it "pond".
  • isoprophlex 2 hours ago
    Great for CI i guess; or is this meant as a cost savings measure where you can swap out your databricks for self-hosted this?
  • PIYUSHMSONI98 45 minutes ago
    [flagged]