Comment by Florian on What's the proper denomination for NA in pyspark...
I personally would use NaN for numeric columns and a string not applicable for string columns, and use None when the data is actually missing so you are able to distinguish between those. But that is...
View ArticleComment by Florian on How to use uploaded file for saving onto local server...
Hi @mikey, what function would you only want to run after an actionButton is clicked? Also, the answer to your question will probably be a bit too elaborate for the comments, so please consider opening...
View ArticleComment by Florian on How to create a countdown timer in Shiny?
@Jamie That depends on the computational complexity of the function. If it's something as simple as this, I do not expect it to be a problem. However, over a longer period of time, the timer might...
View ArticleComment by Florian on Finding non-numeric data in a data frame or vector
Thanbks @Ashish and Ben. I think checking for decimals and negative numbers is a good idea, however the resulting regex Ben proposed will also match invalid elements. So the answer now includes a regex...
View ArticleComment by Florian on Delete unused packages from requirements file
I recently released deptry, a command line utility that checks for various issues with a project's dependencies, such as obsolete, missing or transitive dependencies. It supports requirement.txt files...
View ArticleComment by Florian on Delete unused packages from requirements file
Thanks @pranavcode for the feedback. For now, you can ignore those by running deptry as deptry . -io util,views, or adding configuration to pyproject.toml. Could you elaborate why they are false...
View ArticleComment by Florian on Python code checker for modules included in...
@Brian I recently tried to develop a tool that identifies unused packages, which is also able to deal with the differences in PyPi package names and the imported module names; see my answer here if you...
View ArticleComment by Florian on Nodepool in AKS not scaling down to 0 nodes?
Thanks, that is a good suggestion to give a try! However, apparently that is not allowed on nodepools where the autoscaler is enabled: Cannot scale cluster autoscaler enabled node pool.
View ArticleComment by Florian on Nodepool in AKS not scaling down to 0 nodes?
Thanks for your answer! I am not sure I understand it 100% though. In Azure, the default nodepool is a system nodepool by default. See here. Is this what you refer to a a "dedicated system node pool"?...
View ArticleComment by Florian on Nodepool in AKS not scaling down to 0 nodes?
Thanks for your reply! I see that this could work to scale down the node pool manually occasionally, but in my scenario I want the node pool to scale down to 0 automatically regularly.
View ArticleComment by Florian on deptry reports dependencies as unused, although I need...
I think this actually seems to be an issue with dogpile-cache and not with deptry. When you install a package with an extras option, that should install the base package plus the additional...
View ArticleAnswer by Florian for Pyspark: Convert Column from String Type to Timestamp Type
You use capital M to identify the months as well as the minutes; the minutes should be identified with m, see here. A working example using to_timestamp is given below, hope this helps!import...
View ArticleAnswer by Florian for Problem using 'window' function to group by day in PySpark
I believe you do not need to use a Window to achieve what you want. You would need this for example if you want to have some aggregation of the days before each given date. In your example, you could...
View ArticleAnswer by Florian for SQL/PySpark: Create a new column consisting of a number...
Here is one possible solution, I added some sample data. It indeed uses a window function, as you suggested yourself. Hope this helps!import pyspark.sql.functions as Ffrom pyspark.sql.window import...
View ArticleAnswer by Florian for How to use grepl in every row?
With your new sample data, you could do:pat <- c("mary", "john", "anthony") text <- c("This is a long text about anthony", "This is another long text about john", "This is a final text about...
View ArticleAnswer by Florian for Updating the choices in pickerInput 2 based on the...
You can use uiOutput and renderUI to dynamically generate the required input object. A working example is given below, hope this helps.library(shiny)library(shinyWidgets)managers <- c('Ram',...
View ArticleAnswer by Florian for Insert Pyspark Python k-means model prediction into DF...
You could use map to get the first entry of your rdd with the features, and then use zip to add the predicted clusters. You can convert the resulting rdd with createDataFrame. An example is shown...
View ArticleDifference in environment when using 'docker build' or 'docker run'
I am starting to learn Docker, and I ran into some behavior I could not understand.I want to use conda inside my Docker container, but when using docker build . I ran into the error /bin/sh: 1: conda:...
View ArticleClik here to view.
Answer by Florian for How to change label color of textInput in Shiny Dashboard
In the case of the dateInput, the id is given to a div that wraps both the label and the input itself. In the case of the textInput however, the id is only passed to the input itself, and not to a div...
View ArticleAnswer by Florian for Show modal onclick plotly bar plot
You can use an observer and the event_data from your scatterchart to accomplish that. Working example below, hope this helps!library(shiny)library(plotly)df1 <- data.frame(x = 1:10, y = 1:10)df2...
View Article
