Is there a difference in Views when started/tested locally vs from Kalix?

1

hi,

I’m building a view with this query SQL code:
query: “SELECT * AS jobs FROM jobs where deleted = false ORDER BY date DESC”

When I start it locally, it works and the jobs get sorted by date.

When I deploy it to Kalix, it does not work anymore and I get this:

ERROR:
Code: Internal
Message:

I have seen the Note in the Kalix doc View page saying : “Some orderings may be rejected, if the view index cannot be efficiently ordered. Generally, to order by a column it should also appear in the WHERE conditions.”

But why is it working locally (different type of database for test ?) and how can I fix that?

Thanks in advance,

Bernard

I have seen the comment in the

2

I have tried to fix the problem modifying the query like that:

query: “SELECT * AS jobs FROM jobs where deleted = false AND date IS NOT NULL ORDER BY date DESC”

Again, that works locally but not when deployed to Kalix.

though, the error is a bit different:
ERROR:
Code: Internal
Message: Incompatible view

I guess it’s related to the underlying data store?

3

The local proxy uses H2 for convenience, so there may be some small differences that can explain why it works in a local proxy but not in an actually deployed service. The intention is to not let such queries pass through in the local proxy if they will not work in production.

Surprising that the query does not work though, we’ll have to investigate that, what is the type of the date field in the message you store in the view? Sounds like a bug to me.

The second error you get is because of deploying a new version of the view which is not 1:1 with the previous, to not accidentally drop user data, or cause huge amounts of re-processing, for such scenarios the view is instead failed. The easiest way to handle it is to give the changed query a new view name.

We are working on backoffice support to drop the previously stored view, when a user considers that the right thing to do, but I think we didn’t release a CLI with the support in place quite yet.

4

Thanks for the explanation, now I get why it keeps failing whatever code I put in the SQL view…

Regarding the date type it’s a google.protobuf.Timestamp date = 1;

5

A couple of related questions:

  1. for any new SQL View (basically any difference in the SQL code) we need to create a new view.proto file, new implementation, etc. and use another table?

  2. When using event sourcing, and creating later on a new view, how do I apply all the events that have happened before to the new view?

In general, it would be nice to get a bit more information on how things are working under the hood in Kalix to make the right design decisions.

Thanks!

6
  1. for any new SQL View (basically any difference in the SQL code) we need to create a new view.proto file, new implementation, etc. and use another table?

Yes, if the query cannot be satisfied with the same underlying Kalix view indexes and tables, you either need to use a new table/view id, or manually drop the existing view data (will soon be possible through the Kalix CLI).

  1. When using event sourcing, and creating later on a new view, how do I apply all the events that have happened before to the new view?

This is what a new view always does for event sourcing, it starts from the first event and works its way to “now”.

1 Like
7

For the record, inspecting and dropping views is now possible, since a while back, through the kalix service views commands list and drop.

1 Like