"not equal" querying

[spleshakov]

Hello,

I went through the documentation and couldn't find a way to filter/query dataframe by "not equal" operator, nor I can think of any workarounds

My task is to filter out rows which SERVICE_AREA_ID value starts with MNS

╔═══╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗
║   │ SERVICE_AREA_ID   │ ZIPCODE           │ COUNTY_CODE       │ COUNTY_NAME       ║
╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢
║ 0 │ MNS001            │ 55001             │ 27163             │ Washington        ║
╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢
║ 1 │ MNS002            │ 55001             │ 27163             │ Washington        ║
╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢
║ 2 │ NaN               │ 53001             │ 55117             │ Sheboygan         ║
╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢
║ 3 │ AC North and WI   │ 54856             │ 52030             │ Bayfield          ║
╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢
║ 4 │ North and WI      │ 54856             │ 52030             │ Bayfield          ║
╚═══╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝

There can be more of these service area than 2, but for now I'm even ready to hardcode these two o query for values other than 01 and 02

P.S. I need to exclude NaN values as well

[spleshakov]

One way I can think of actually, is

• get all values as array
• exclude the two
• loop for remaining values and create a dataframe for each
• concatenate dataframes

[risenW]

Works but sadly, too hacky. We have to support this internally.
We need a filter function that can accept multiple logical operators and also be able to filter by a boolean mask, just like pandas as shown here. There are numerous use cases of this, and we certainly have to implement it.

[risenW]

Created an issue here: #180
Wanna try working on this?

[spleshakov]

I would love to, but I'm not a developer :) all I know is how to report bugs haha

[risenW]

😂 No probs, I’ll probably work on it over the weekend. Thanks for spotting these bugs.

[risenW]

This is now supported in the latest version v0.3.0

you can call iloc, loc and query with conditions. For example, this works in query:

df.query({ condition: df[“A”].ge(5) })

// Chaining multiple conditions work too:

df.query({ condition: df[“A”].ge(5).and(df[“B”].eq(50) })

An array of boolean mask/values of the same length as the DataFrame can be used as well.

cc @spleshakov