Skip to content

When a new node joins, the cache is not copied (Replicated) #231

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
cospin opened this issue Sep 4, 2024 · 2 comments
Open

When a new node joins, the cache is not copied (Replicated) #231

cospin opened this issue Sep 4, 2024 · 2 comments

Comments

@cospin
Copy link

cospin commented Sep 4, 2024

Hello, I am new to Nebulex and I'm testing the Replicated adapter, but I notice that when a new node joins, it does not copy the cache from the other node. In production, DNSCluster is used to connect the nodes dynamically, but in local tests, I'm simply using Node.connect/1.

This is my setup:

defmodule Core.ReplicatedCache do
  use Nebulex.Cache,
    otp_app: :core,
    adapter: Nebulex.Adapters.Replicated
end
config :core, Core.ReplicatedCache,
  primary: [
    gc_interval: :timer.hours(12),
    gc_memory_check_interval: :timer.seconds(10),
    max_size: 1_000_000,
    allocated_memory: 2_000_000_000
  ]
def start(_type, _args) do
    children = [
      Core.Repo,
      {DNSCluster, query: Application.get_env(:core, :dns_cluster_query) || :ignore},
      {Phoenix.PubSub, name: Core.PubSub},
      {Core.ReplicatedCache, []},
      Core.Telemetry,
      Core.Endpoint
    ]
    opts = [strategy: :one_for_one, name: Core.Supervisor]
    Supervisor.start_link(children, opts)
end

I'm starting one local node and creating a new cache record:

iex --name node1@127.0.0.1 --cookie mycookie -S mix

Core.ReplicatedCache.put("key", "value2")
:ok

Then I join a second node, where it can't access the already created key:

iex --name node2@127.0.0.1 --cookie mycookie -S mix

Node.connect(:"node1@127.0.0.1")
true

# key not found, so no cache copy?
Core.ReplicatedCache.get("key")
nil

But new writes works fine between the two:

# put newkey from any node
Core.ReplicatedCache.put("newkey", "value2")
:ok

# get works on both nodes:
Core.ReplicatedCache.get("newkey")
"value2"

So, my conclusion is that when a new node joins, it is not copying the cache.
Am I overlooking something, or am I doing something wrong?

Thanks!

Elixir 1.17.2 (compiled with Erlang/OTP 27)

@cospin
Copy link
Author

cospin commented Sep 4, 2024

Ok, I did a local test but this time adding libcluster, and it seems to work well :)
I guess the problem was my rudimentary use of the cluster with Node.connect/1 (I'm also new to clustering).

I didn't see any of this in the documentation, so, pretty sure I'm doing something wrong with the cluster formation. Any insights? 😅 Anyways, seems like libcluster is a nice addon.

@cabol
Copy link
Owner

cabol commented Oct 19, 2024

Hey! Yeah, the documentation doesn't mention it, but the distributed adapters rely on "Distributed Erlang/Elixir" and they need the cluster properly set up to work correctly. And yeah, libcluster is the way to go. I agree it would be good to have some documentation about it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants