This is forum for Power tool stack (Power Pivot, Power Query, Power BI etc).
What you have is standard pivot table.
You can't do that sort of Avg operation in traditional pivot. Source data granularity must match that of your Rows field. Meaning you need source data aggregated at daily level.
If you use Power Pivot, you could write DAX measure to handle different calculation based on evaluation context. Using IF(HASONEVALUE(column), Calculation1, Calculation2) DAX pattern.